# Spectra Detect Documentation > Enterprise file analysis platform documentation including deployment, API, and configuration. This file contains all documentation content in a single document following the llmstxt.org standard. ## Getting Started with Spectra Detect — First Login, SDM Setup, and File Analysis # Getting started with Spectra Detect This guide walks you through logging in to Spectra Detect Manager (SDM), connecting to the Spectra Intelligence cloud, and running your first file analysis workflow. **What you'll accomplish:** - Log in to Spectra Detect Manager - Connect to the Spectra Intelligence cloud - Configure your first scan input - Submit a file and view analysis results ## Prerequisites Before you begin: - Access to your organization's Spectra Detect deployment (or deployment credentials if setting up from scratch) - Familiarity with your chosen deployment model (OVA/AMI, K8s Mono, or K8s Micro) - For new deployments: access to the appropriate infrastructure (VMware/AWS for OVA/AMI, Kubernetes cluster for K8s deployments) **Tip: Initial credentials** Your initial SDM administrator username and password are provided by [ReversingLabs Support](mailto:support@reversinglabs.com). Change the default password after your first login. ## Step 1: Access your deployment Spectra Detect is available in two deployment models (OVA/AMI, K8s Micro). This guide assumes you have already deployed Spectra Detect or have access to an existing deployment. For deployment instructions, see: - [AWS EKS Micro Deployment](./Deployment/AWS-EKS-Deployment-Micro/index.md) - [On-Premises Deployment](./Deployment/index.md) ## Step 2: Log in to Spectra Detect Manager Spectra Detect Manager (SDM) is the central web interface for configuring Workers, monitoring analysis activity, and managing the cluster. It is available in OVA/AMI and K8s Micro deployments. 1. Open your browser and navigate to the SDM URL provided by your administrator (for example, `https://sdm.example.com`). 2. Log in with the administrator credentials provided by [ReversingLabs Support](mailto:support@reversinglabs.com). 3. Change the default password when prompted. **Note: SDM is not included in the K8s Micro deployment (v5.7). Refer to the [AWS EKS Micro Deployment](./Deployment/AWS-EKS-Deployment-Micro/index.md) guide for configuration in that model.** ## Step 3: Configure the Spectra Intelligence cloud connection Spectra Detect Manager must be connected to [Spectra Intelligence](/SpectraIntelligence/) to receive system updates, appliance upgrades, and cloud-enriched classification data. Without this connection, software updates cannot be delivered automatically and Deep Cloud Analysis will not be available. 1. In SDM, navigate to **Administration > Spectra Detect Manager**. 2. Scroll to the **Spectra Intelligence** section. 3. Select the **Enable Spectra Intelligence** checkbox. 4. Enter your Spectra Intelligence **username** and **password**. 5. Select **Save**. SDM will restart and begin polling Spectra Intelligence every 60 minutes. If the connection fails, verify that your network allows outbound HTTPS (port 443) to `appliance-api.reversinglabs.com`. See [Network Ports](./Admin/ManagerSettings.md#network-ports) for the full list of required ports. **Note: If you do not yet have Spectra Intelligence credentials, contact [ReversingLabs Support](mailto:support@reversinglabs.com).** ## Step 4: Configure a scan input Configure at least one file input source so Spectra Detect knows where to pick up files for analysis. 1. In SDM, navigate to **Configuration > Analysis Input**. 2. Select your input type (S3 bucket, ICAP, network share, or API submission). 3. Enter connection details and authentication credentials. 4. Select **Save**. For detailed input options, see [Analysis Input Configuration](./Config/AnalysisInput.md). ## Step 5: Submit a file and view results Once a scan input is configured, files submitted through that input are automatically analyzed by [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis). To verify analysis is working: 1. Submit a test file through your configured input. 2. Open the [Dashboard](./Usage/Dashboard.md) in SDM. 3. Confirm the file appears with a classification (Malicious, Suspicious, Goodware, or Unknown). 4. Select the file to view its full analysis report. ## Step 6: Configure YARA rules (optional) Spectra Detect supports custom [YARA rules](./Usage/YARA.md) for detection of specific file patterns. Rules can be uploaded through SDM and are automatically synchronized across all connected Workers. ## Next steps Now that you've configured your first scan input and verified analysis is working, explore the full capabilities of Spectra Detect: - **[Appliance Configuration](./Config/ApplianceConfiguration.md)** - Network and system settings - **[Manager Settings](./Admin/ManagerSettings.md)** - SDM configuration reference - **[Dashboard](./Usage/Dashboard.md)** - Monitoring cluster status and throughput - **[Management API](./API/ManagementAPI.md)** - Automate Spectra Detect via REST API - **[Updating](./Admin/Updating.md)** - Software update procedures --- ## Spectra Detect — Enterprise File Analysis and Malware Detection Platform # Spectra Detect — Enterprise File Analysis and Malware Detection Spectra Detect is a file analysis system available in two deployment configurations: - **OVA/AMI deployment**: traditional virtual machine deployment with Workers, Hubs, and Spectra Detect Manager (SDM). - **K8s Micro deployment**: redesigned architecture where traditional Workers and Hubs are decomposed into specialized microservices. - Workers are broken down into individual processing components. - Hub functionality is currently supported only for S3 connector integration. - SDM is not included in this preview release. ## About Spectra Detect Spectra Detect uses a flexible cluster architecture that scales incrementally to support distributed or centralized file processing across physical and cloud environments. The cluster can incrementally scale file processing capacity from 100K to 100M files per day by adding Worker nodes. Spectra Detect is an enterprise-scale automated malware detection platform built for organizations that need to inspect millions of files per day across email gateways, file shares, S3 buckets, ICAP proxies, and other ingestion points — without creating bottlenecks or slowing down production workflows. Powered by [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis), it performs deep static analysis on over 400 file formats, identifying malware, suspicious indicators, and embedded threats in seconds per file. Analysis results include full indicator extraction, threat names, risk scores, and mapping to [MITRE ATT&CK](https://attack.mitre.org/) tactics — all delivered without executing files. ## File analysis The platform scales horizontally from 100,000 to 100 million files per day by adding Worker nodes. In virtual machine deployments (OVA/AMI), Workers can be provisioned manually to match capacity needs. In Kubernetes deployments (K8s Micro), Workers auto-scale based on queue depth. Results feed directly into existing security infrastructure — SIEM, SOAR, EDR, and threat intelligence platforms — through webhooks, S3 output, and REST APIs. Enterprise YARA rule deployment and synchronization across all Workers is managed centrally through Spectra Detect Manager. In OVA/AMI deployments, every Worker contains an instance of [Spectra Core](https://www.reversinglabs.com/products/spectra-core), a platform for automated static decomposition and analysis of files. ## Architecture In K8s Micro deployment, the Spectra Core functionality is distributed across multiple microservice components that work together to analyze files. Spectra Detect uses a three-tier cluster architecture: a central manager for control and configuration, one or more Hubs for ingestion and load distribution, and horizontally scalable Workers that perform the analysis. ![Spectra Detect architecture](./images/spectra-detect-architecture.png) Spectra Core can automatically unpack and extract information from more than 300 PE packers, archives, installation packages, firmware images, documents, and mobile application formats. The extracted information includes metadata such as strings, format header details, function names, library dependencies, file segments, and capabilities. This information is contained in the Worker analysis report (JSON file). ### Components **Spectra Detect Manager (SDM)** SDM is the central control plane for the entire cluster. It provides a web UI and REST API for monitoring appliance health, managing licenses, deploying software updates, and synchronizing YARA rules across all connected Workers and Spectra Analyze appliances. SDM connects to [Spectra Intelligence](/SpectraIntelligence/) for cloud-enriched classifications and automatic update delivery. Spectra Detect Manager (SDM) is a management platform that provides a centralized view of the status of ReversingLabs appliances, centralized software upgrades, configuration of authorized appliances, and YARA rules deployment. SDM is available in OVA/AMI deployments. It is not included in the K8s Micro preview (v5.7). **Info: SDM is available in OVA/AMI deployments, but is **not included** in the K8s Micro deployment preview (v5.7).** **Hub** The Hub is the ingestion and distribution layer between file sources and Workers. It receives files from configured input connectors — S3 buckets, file shares, ICAP proxies, email gateways, and direct API submissions — and distributes them to available Workers for analysis. In high-availability configurations, two Hubs can be deployed for redundancy. **Workers** Workers perform the actual file analysis using [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis). Each Worker unpacks and inspects submitted files, extracts indicators, assigns a risk score, and returns the result to the Hub. Workers scale horizontally: additional Workers are added manually in OVA/AMI deployments or provisioned automatically by Kubernetes based on queue depth. ### Deployment models | Model | Description | Scaling | |---|---|---| | OVA/AMI | Virtual machine with Workers, Hubs, and SDM | Manual — provision new VMs | | K8s Micro | Microservices architecture on Kubernetes | Automatic — per-component scaling | The Manager functions as a mediator between ReversingLabs appliances connected to it. When YARA rulesets are uploaded to any of the connected appliances that support them, the Manager ensures the rulesets are synchronized across all applicable appliances. ### Multi-region deployment In multi-region deployments, global load balancers are added in front of each tier to distribute traffic across geographic locations and provide fault tolerance. - Status overview for multiple ReversingLabs product types - License management for connected Spectra Analyze and Spectra Detect Worker appliances - Control for upgrading Spectra Analyze, Spectra Detect Worker and Hub - Centralized YARA rules deployment and synchronization between Spectra Analyze, and Spectra Detect Worker - Alerts for critical system services - Support for sample search across all connected and authorized Spectra Analyze appliances - Configuration modules for centralized management of Spectra Analyze, Spectra Detect Worker and Hub - Support for configuring the Connectors service on Spectra Analyze and Spectra Detect appliances A global load balancer sits in front of the SDM cluster, routing management and API traffic to the active SDM instance regardless of which region handles the request. A separate load balancer sits in front of the Hub cluster, distributing file submission traffic across Hubs. Workers are deployed in two or more independent regional clusters — each region runs its own Hub-and-Worker pool, so file analysis stays local to the region where files are submitted. The SDM cluster spans regions for centralized control while Worker clusters remain regionally isolated for performance and data residency. **Components added in multi-region deployments:** - **Global Load Balancer (SDM)** — routes SDM API and UI traffic to the active SDM node; handles failover between SDM instances across regions - **Global Load Balancer (Hub)** — distributes incoming file submissions across regional Hub clusters; can route by geography to keep analysis traffic local - **Regional Hub + Worker clusters** — each region runs an independent Hub-and-Worker pool; Workers in each region auto-scale independently based on local queue depth ## Documentation ### Getting started - [Getting started](getting-started.md) — first login, cloud connection, and first analysis ### Deployment - [AWS EKS Micro Deployment](./Deployment/AWS-EKS-Deployment-Micro/index.md) — microservices architecture deployment on AWS EKS - [On-Premises Deployment](./Deployment/index.md) — hardware and virtual machine deployment options ### Configuration - [Appliance Configuration](./Config/ApplianceConfiguration.md) — network, storage, and system settings - [Analysis Input](./Config/AnalysisInput.md) — configure file sources and input connectors - [YARA Sync](./Admin/YARASync.md) — synchronize YARA rules across appliances - [Certificate Management](./Admin/CertificateManagement.md) — TLS/SSL certificate configuration ### Usage - [Dashboard](./Usage/Dashboard.md) — monitoring cluster status and analysis results - [Analysis](./Usage/Analysis.md) — understanding analysis results and classifications - [YARA Rules](./Usage/YARA.md) — managing and deploying YARA rulesets ### API - [Management API](./API/ManagementAPI.md) — REST API for appliance management and automation ### Administration - [Manager Settings](./Admin/ManagerSettings.md) — Spectra Detect Manager configuration reference - [Updating](./Admin/Updating.md) — software update procedures - [Troubleshooting](./troubleshooting.md) — common issues and solutions --- ## Spectra Detect AWS EKS Config Reference — Secrets and ConfigMap Values ## Detect Platform General Values | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | connectorS3.enabled | bool | `false` | If enabled, connector-s3 chart will be used and S3 connector pods will be deployed. | | global.applyRestrictedPolicy | bool | `false` | Apply needed parts of the restricted policy to all sub-charts. When enabled, SDM needs to be disabled, since currently it is only supported for Worker. | | global.clusterDomainName | string | `"cluster.local"` | Cluster Domain Name | | global.postgresCustomSecretName | string | `nil` | Postgres custom secret name. If not set, default secret name will be used. | | global.rabbitmqAdminCustomSecretName | string | `nil` | RabbitMQ custom admin secret name. If not set, default secret name will be used. | | global.rabbitmqCustomSecretName | string | `nil` | RabbitMQ custom secret name. If not set, default secret name will be used. | | global.umbrella | bool | `true` | Notifies dependency charts that they are being used from the umbrella chart. Must always be set to true. | | global.useReloader | boolean | `true` | Whether to enable Reloader annotations. | | logging.enabled | bool | `false` | If enabled, logging stack (Alloy + Loki + Grafana) will be deployed for log collection and visualization. | | prometheus.enabled | bool | `true` | Has to be enabled when Prometheus is used. It creates a secret containing the Prometheus configuration, enabling other charts to connect. | | registry.authSecretName | string | `"rl-registry-key"` | The name of a Kubernetes secret resource used for pulling images from a Docker registry. | | registry.authSecretPassword | string | `nil` | The password used for authenticating with the registry. | | registry.authSecretUsername | string | `nil` | The username used for authenticating with the registry. | | registry.createRegistrySecret | bool | `true` | If enabled, a Kubernetes secret for pulling images will be created. If disabled, the secret must be created manually in the namespace. | | registry.imageRegistry | string | `"registry.reversinglabs.com"` | The image registry address. | | reloader.enabled | bool | `true` | If enabled, reloader will be deployed. Reloader ensures that latest configuration is applied at all time. | | sdm.enabled | bool | `false` | If enabled, Spectra Detect Manager chart will be deployed. | | worker.enabled | bool | `true` | If enabled, worker chart will be used and Worker service pods will be deployed. | ## Processing ### Worker Secrets | Secret (when custom secret name is used) | Secret (default secret name) | Type | Description | Used in deployments (Pods) | | :---- | :---- | :---- | :---- | :---- | | `` | `-secret-worker-api-token` | Optional | Token secret which contains token that is used to protect all endpoints with /api/ prefix, e.g. file upload. | Auth | | `` | `-secret-worker-api-task-token` | Optional | Token secret which contains token that is used to protect /api/tiscale/v1/task endpoints. If left empty, the mentioned API is protected by `-secret-worker-api-token` | Auth | | `` | `-secret-worker-cloud` | Required when related feature is enabled | Basic authentication secret which contains username and password for Spectra Intelligence authentication. Required when Spectra Intelligence is enabled (configuration.cloud.enabled). | Processor, Retry Processor, Preprocessor, Postprocessor, Receiver | | `` | `-secret-worker-cloud-proxy` | Required when related feature is enabled | Basic authentication secret which contains username and password for Spectra Intelligence Proxy authentication. Required when Spectra Intelligence Proxy is enabled (configuration.cloud.proxy.enabled). | Processor, Retry processor, Preprocessor, Postprocessor, Receiver, Cloud Cache | | `` | `-secret-worker-aws` | Required when related feature is enabled | Basic authentication secret which contains username and password for AWS authentication. Required if any type of S3 storage (File, SNS, Report, Unpacked) is enabled (configuration.s3.enabled, configuration.sns.enabled, configuration.reportS3.enabled, configuration.unpackedS3.enabled) | Postprocessor | | `` | `-secret-worker-azure` | Required when related feature is enabled | Basic authentication secret which contains username and password for Azure authentication. Required if any type of ADL storage (File, Report, Unpacked) is enabled (configuration.adl.enabled, configuration.reportAdl.enabled, configuration.unpackedAdl.enabled). | Postprocessor | | `` | `-secret-worker-ms-graph` | Required when related feature is enabled | Basic authentication secret which contains username and password for Microsoft Cloud Storage authentication. Required if any type of Microsoft Cloud storage (File, Report, Unpacked) is enabled (configuration.msGraph.enabled, configuration.reportMsGraph.enabled, configuration.unpackedMsGraph.enabled). | Postprocessor | | `` | `-secret-worker-unpacked-s3` | Optional | Secret which contains only password. Used for encryption of the archive file. Relevant only when the configuration.unpackedS3.archiveUnpacked option is set to true. | Postprocessor | | `` | `-secret-worker-report-s3` | Optional | Secret which contains only password. Used for encryption of the archive file. Relevant only when the configuration.reportS3.archiveSplitReport option is set to true. | Postprocessor | | `` | `-secret-worker-unpacked-adl` | Optional | Secret which contains only password. Used for encryption of the archive file. Relevant only when the configuration.unpackedAdl.archiveUnpacked option is set to true. | Postprocessor | | `` | `-secret-worker-report-adl` | Optional | Secret which contains only password. Used for encryption of the archive file. Relevant only when the configuration.reportAdl.archiveSplitReport option is set to true. | Postprocessor | | `` | `-secret-worker-unpacked-ms-graph` | Optional | Secret which contains only password. Used for encryption of the archive file. Relevant only when the configuration.unpackedMsGraph.archiveUnpacked option is set to true. | Postprocessor | | `` | `-secret-worker-report-ms-graph` | Optional | Secret which contains only password. Used for encryption of the archive file. Relevant only when the configuration.reportMsGraph.archiveSplitReport option is set to true. | Postprocessor | | `` | `-secret-worker-splunk` | Optional | Token secret which contains token for Splunk authentication. Relevant only if Splunk Integration is enabled (configuration.splunk.enabled). | Postprocessor | | `` | `-secret-worker-archive-zip` | Optional | Secret which contains only password. Relevant only when the configuration.archive.fileWrapper value is set to "zip" or "mzip". | Postprocessor | | `` | `-secret-worker-spectra-analyze-integration-token` | Required when related feature is enabled | Token secret which contains the token used in authentication on Spectra Analyze when Spectra Analyze Integration is enabled (configuration.spectraAnalyzeIntegration.enabled). This token should be created in Spectra Analyze. | Postprocessor | | `` | `-secret-worker-auth-creds` | Optional | Secret of type kubernetes.io/dockerconfigjson. Contains authentication credentials for multiple registries. Required when using API for container image upload. | Receiver | | `` | `-secret-worker-ca-certs` | Optional | Opaque secret which contains a single key, ca_bundle. The value of that key are bundled certificates. Needed when certificates are required, example when using API for container image upload and the registry the image is on requires it. | Receiver | ### Worker Secret Configuration Values | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | secrets.api | object | `-` | Secret configuration values for setting the authorization token that is used to protect all endpoint with /api/ prefix. | | secrets.api.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `token` parameter. WARNING: Use this for convenience/testing only. | | secrets.api.customSecretName | string | `nil` | Custom secret name. If not set, default secret name will be used. | | secrets.api.token | string | `""` | API token secret. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.apiTask | object | `-` | Secret configuration values for setting the authorization token that is used to protect /api/tiscale/v1/task endpoints. | | secrets.apiTask.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `token` parameter. WARNING: Use this for convenience/testing only. | | secrets.apiTask.customSecretName | string | `nil` | Custom secret name. If not set, default secret name will be used. | | secrets.apiTask.token | string | `""` | API task token secret. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.archive | object | `-` | Secret configuration values for setting the password used for encryption of the zip file. Relevant only when the configuration.archive.fileWrapper value is set to 'zip' or 'mzip'. | | secrets.archive.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `password` parameter. WARNING: Use this for convenience/testing only. | | secrets.archive.customSecretName | string | `nil` | Custom secret name. If not set, default secret name will be used. | | secrets.archive.password | string | `""` | Archive password secret. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.authCreds | object | `-` | Secret configuration values for setting the dockerconfig secret which will contain registry authentication credentials used for upload of the container images for analysis. | | secrets.authCreds.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret of type dockerconfig using the `credentials` list of `username`, `password` and `registry` parameters. WARNING: Use this for convenience/testing only. | | secrets.authCreds.credentials | list | `[]` | List of credentials from which the dockerconfig secret can be created. Only used if 'createUserSecret' is set to 'true'. Structure of the list items can be found here: [Authentication Credentials](#worker-secret-configuration-values---authentication-credentials). WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.authCreds.customSecretName | string | `nil` | Custom secret name. If not set, default secret name will be used. | | secrets.aws | object | `-` | Secret configuration values for setting the credentials for any S3 type storage. | | secrets.aws.awsS3AccessKeyId | string | `""` | AWS S3 access key ID. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.aws.awsS3SecretAccessKey | string | `""` | AWS S3 secret access key. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.aws.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `awsS3AccessKeyId` and `awsS3SecretAccessKey` parameters. WARNING: Use this for convenience/testing only. | | secrets.aws.customSecretName | string | `nil` | Custom secret name. If not set, default secret name will be used. | | secrets.azure | object | `-` | Secret configuration values for setting the Azure credentials. | | secrets.azure.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `username` and `password` parameters. WARNING: Use this for convenience/testing only. | | secrets.azure.customSecretName | string | `nil` | Custom secret name. If not set, default secret name will be used. | | secrets.azure.password | string | `""` | Azure password secret. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.azure.username | string | `""` | Azure username secret. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.caCerts | object | `-` | Secret configuration values for setting the secret which will contain bundled certificates. | | secrets.caCerts.certificates | list | `[]` | List of certificates which will be bundled. Each list item must contain the complete PEM-encoded file content of the certificate, including headers, not its file path. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive data in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.caCerts.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret which contains bundled certificates that are given in the `certificates` list. WARNING: Use this for convenience/testing only. | | secrets.caCerts.customSecretName | string | `nil` | Custom secret name. If not set, default secret name will be used. | | secrets.cloud | object | `-` | Secret configuration values for setting the Spectra Intelligence credentials. | | secrets.cloud.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `username` and `password` parameters. WARNING: Use this for convenience/testing only. | | secrets.cloud.customSecretName | string | `nil` | Custom secret name. If not set, default secret name will be used. | | secrets.cloud.password | string | `""` | Cloud password secret. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments.. | | secrets.cloud.username | string | `""` | Cloud username secret. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.cloudProxy | object | `-` | Secret configuration values for setting the Spectra Intelligence credentials when proxy is used. | | secrets.cloudProxy.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `username` and `password` parameters. WARNING: Use this for convenience/testing only. | | secrets.cloudProxy.customSecretName | string | `nil` | Custom secret name. If not set, default secret name will be used. | | secrets.cloudProxy.password | string | `""` | Cloud proxy password secret. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.cloudProxy.username | string | `""` | Cloud proxy username secret. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.msGraph | object | `-` | Secret configuration values for setting the credentials for Microsoft Cloud Storage. | | secrets.msGraph.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `username` and `password` parameters. WARNING: Use this for convenience/testing only. | | secrets.msGraph.customSecretName | string | `nil` | Custom secret name. If not set, default secret name will be used. | | secrets.msGraph.password | string | `""` | MS Graph password secret. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.msGraph.username | string | `""` | MS Graph username secret. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.reportAdl | object | `-` | Secret configuration values for setting the password used for encryption of the archive file. Relevant only when the configuration.reportAdl.archiveSplitReport option is set to true. | | secrets.reportAdl.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `password` parameter. WARNING: Use this for convenience/testing only. | | secrets.reportAdl.customSecretName | string | `nil` | Custom secret name. If not set, default secret name will be used. | | secrets.reportAdl.password | string | `""` | Report ADL password secret. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.reportMsGraph | object | `-` | Secret configuration values for setting the password used for encryption of the archive file. Relevant only when the configuration.unpackedMsGraph.archiveSplitReport option is set to true. | | secrets.reportMsGraph.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `password` parameter. WARNING: Use this for convenience/testing only. | | secrets.reportMsGraph.customSecretName | string | `nil` | Custom secret name. If not set, default secret name will be used. | | secrets.reportMsGraph.password | string | `""` | Report MS Graph password secret. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.reportS3 | object | `-` | Secret configuration values for setting the password used for encryption of the archive file. Relevant only when the configuration.reportS3.archiveSplitReport option is set to true. | | secrets.reportS3.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `password` parameter. WARNING: Use this for convenience/testing only. | | secrets.reportS3.customSecretName | string | `nil` | Custom secret name. If not set, default secret name will be used. | | secrets.reportS3.password | string | `""` | Report S3 password secret. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.spectraAnalyzeIntegration | object | `-` | Secret configuration values for setting the token used for authentication on Spectra Analyze when Spectra Analyze Integration is enabled (configuration.spectraAnalyzeIntegration.enabled). | | secrets.spectraAnalyzeIntegration.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `token` parameter. WARNING: Use this for convenience/testing only. | | secrets.spectraAnalyzeIntegration.customSecretName | string | `nil` | Custom secret name. If not set, default secret name will be used. | | secrets.spectraAnalyzeIntegration.token | string | `""` | Spectra Analyze Integration token secret. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.splunk | object | `-` | Secret configuration values for setting the token for Splunk authentication. | | secrets.splunk.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `token` parameter. WARNING: Use this for convenience/testing only. | | secrets.splunk.customSecretName | string | `nil` | Custom secret name. If not set, default secret name will be used. | | secrets.splunk.token | string | `""` | Splunk token secret. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.unpackedAdl | object | `-` | Secret configuration values for setting the password used for encryption of the archive file. Relevant only when the configuration.unpackedAdl.archiveUnpacked option is set to true. | | secrets.unpackedAdl.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `password` parameter. WARNING: Use this for convenience/testing only. | | secrets.unpackedAdl.customSecretName | string | `nil` | Custom secret name. If not set, default secret name will be used. | | secrets.unpackedAdl.password | string | `""` | Unpacked ADL password secret. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.unpackedMsGraph | object | `-` | Secret configuration values for setting the password used for encryption of the archive file. Relevant only when the configuration.reportMsGraph.archiveSplitReport option is set to true. | | secrets.unpackedMsGraph.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `password` parameter. WARNING: Use this for convenience/testing only. | | secrets.unpackedMsGraph.customSecretName | string | `nil` | Custom secret name. If not set, default secret name will be used. | | secrets.unpackedMsGraph.password | string | `""` | Unpacked MS Graph password secret. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.unpackedS3 | object | `-` | Secret configuration values for setting the password used for encryption of the archive file. Relevant only when the configuration.unpackedS3.archiveUnpacked option is set to true. | | secrets.unpackedS3.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `password` parameter. WARNING: Use this for convenience/testing only. | | secrets.unpackedS3.customSecretName | string | `nil` | Custom secret name. If not set, default secret name will be used. | | secrets.unpackedS3.password | string | `""` | Unpacked S3 password secret. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | ### Worker Secret Configuration Values - Authentication Credentials | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | credentials[n] | list | `-` | List of credentials to be added to the dockerconfig secret. All values need to be set for the secret to be created. | | credentials[n].password | string | `""` | Registry password. WARNING: Use this for convenience/testing only. Do not store sensitive data in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | credentials[n].registry | string | `""` | Registry name. | | credentials[n].username | string | `""` | Registry username. WARNING: Use this for convenience/testing only. Do not store sensitive data in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | ### Worker Application Configuration Values | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | appliance.configMode | string | `"STANDARD"` | Configuration mode of the appliance. Allowed values: CONFIGMAP (Configuration is provided with configmap), STANDARD (configuration is provided over UI). | | configuration.a1000 | object | `-` | Integration with Spectra Analyze appliance. | | configuration.a1000.host | string | `""` | The hostname or IP address of the A1000 appliance associated with the Worker. | | configuration.adl | object | `-` | Settings for storing files in an Azure Data Lake container. | | configuration.adl.container | string | `""` | The hostname or IP address of the Azure Data Lake container that will be used for storage. Required when storing files in ADL is enabled. | | configuration.adl.enabled | bool | `false` | Enable or disable the storage of processed files. | | configuration.adl.folder | string | `""` | Specify the name of the folder on the container where files will be stored. | | configuration.apiServer | object | `-` | Configures a custom Worker IP address which is included in the response when uploading a file to the Worker for processing. | | configuration.apiServer.host | string | `""` | Configures the hostname or IP address of the Worker. Only necessary if the default IP address or network interface is incorrect. | | configuration.archive | object | `-` | After processing, files can be zipped before external storage. Available only for S3 and Azure. | | configuration.archive.fileWrapper | string | `""` | Specify whether the files should be compressed as a ZIP archive before uploading to external storage. Supported values are: zip, mzip. If this parameter is left blank, files will be uploaded in their original format. | | configuration.archive.zipCompress | int | `0` | ZIP compression level to use when storing files in a ZIP file. Allowed range: 0 (no compression) to 9 (maximum compression). | | configuration.archive.zipMaxfiles | int | `0` | Maximum allowed number of files that can be stored in one ZIP archive. Allowed range: 1-65535. 0 represents unlimited. | | configuration.authentication | object | `-` | Authentication settings for Detect Worker. | | configuration.authentication.enabled | bool | `false` | Enable/disable authentication on Detect Worker ingress APIs. | | configuration.authentication.externalAuthUrl | string | `""` | If set external/custom authentication service will be used for authentication, otherwise simple Token service is deployed which protects paths with tokens defined in the secrets. | | configuration.aws | object | `-` | Configuration of integration with AWS or AWS-compatible storage to be used for SNS, and for uploading files and analysis reports to S3. | | configuration.aws.caPath | string | `""` | Path on the file system pointing to the certificate of a custom (self-hosted) S3 server. | | configuration.aws.endpointUrl | string | `""` | Only required in non-AWS setups in order to store files to an S3-compatible server. When this parameter is left blank, the default is `https://aws.amazonaws.com`. Supported pattern(s): `https?://.+"`. | | configuration.aws.maxReattempts | int | `5` | Maximum number of retries when saving a report to an S3-compatible server. | | configuration.aws.payloadSigningEnabled | bool | `false` | Specifies whether to include an SHA-256 checksum with Amazon Signature Version 4 payloads. | | configuration.aws.region | string | `"us-east-1"` | Specify the correct AWS geographical region where the S3 bucket is located. Required parameter, ignored for non-AWS setups. | | configuration.aws.serverSideEncryption | string | `""` | Specify the encryption algorithm used on the target S3 bucket (e.g. aws:kms or AES256). | | configuration.aws.sslVerify | bool | `false` | Enable/disable SSL verification. | | configuration.awsRole | object | `-` | Configures the AWS IAM roles used to access S3 buckets without sharing secret keys. The IAM role which will be used to obtain temporary tokens has to be created in the AWS console. | | configuration.awsRole.enableArn | bool | `false` | Enables or disables this entire feature. | | configuration.awsRole.externalRoleId | string | `""` | The external ID of the role that will be assumed. This can be any string. Usually, it’s an ID provided by the entity which uses (but doesn’t own) an S3 bucket. The owner of that bucket takes that external ID and builds an ARN with it. | | configuration.awsRole.refreshBuffer | int | `5` | Number of seconds to fetch a new ARN token before the token timeout is reached. | | configuration.awsRole.roleArn | string | `""` | The role ARN created using the external role ID and an Amazon ID. In other words, the ARN which allows a Worker to obtain a temporary token, which then allows it to save to S3 buckets without a secret access key. | | configuration.awsRole.roleSessionName | string | `""` | Name of the session visible in AWS logs. Can be any string. | | configuration.awsRole.tokenDuration | int | `900` | How long before the authentication token expires and is refreshed. The minimum value is 900 seconds. | | configuration.azure | object | `-` | Configures integration with Azure Data Lake Gen2 for the purpose of storing processed files in Azure Data Lake containers. | | configuration.azure.endpointSuffix | string | `"core.windows.net"` | Specify the suffix for the address of your Azure Data Lake container. | | configuration.callback | object | `-` | Settings for automatically sending file analysis reports via POST request. | | configuration.callback.advancedFilterEnabled | bool | `false` | Enable/disable the advanced filter. | | configuration.callback.advancedFilterName | string | `""` | Name of the advanced filter. | | configuration.callback.caPath | string | `""` | If the url parameter is configured to use HTTPS, this parameter can be used to set the path to the certificate file. This automatically enables SSL verification. If this parameter is left blank or not configured, SSL verification will be disabled, and the certificate will not be validated. | | configuration.callback.enabled | bool | `false` | Enable/disable connection. | | configuration.callback.maliciousOnly | bool | `false` | When set, the report will only contain malicious and suspicious children. | | configuration.callback.reportType | string | `"medium"` | Specifies which report\_type is returned. By default, or when empty, only the medium (summary) report is provided in the callback response. Set to extended\_small, small, medium or large to view results of filtering the full report. | | configuration.callback.splitReport | bool | `false` | By default, reports contain information on parent files and all extracted children files. If set to true, reports for extracted files will be separated from the full report and saved as standalone files. If any user-defined data was appended to the analyzed parent file, it will be included in every split child report. | | configuration.callback.sslVerify | bool | `false` | Enable/disable SSL verification | | configuration.callback.timeout | int | `5` | Specify the number of seconds to wait before the POST request times out. In case of failure, the Worker will retry the request up to six times, increasing the waiting time between requests after the second retry has failed. With the default timeout set, the total possible waiting time before a request finally fails is 159 seconds. | | configuration.callback.topContainerOnly | bool | `false` | If set to true, the reports will only contain metadata for the top container. Reports for unpacked files will not be generated. | | configuration.callback.url | string | `""` | Specify the full URL that will be used to send the callback POST request. Both HTTP and HTTPS are supported. If this parameter is left blank, reports will not be sent, and the callback feature will be disabled. Supported pattern(s): http?://.+ | | configuration.callback.view | string | `""` | Specifies whether a custom report view should be applied to the report. | | configuration.cef | object | `-` | Configures Common Event Format (CEF) settings. CEF is an extensible, text-based logging and auditing format that uses a standard header and a variable extension, formatted as key-value pairs. | | configuration.cef.cefMsgHashType | string | `"md5"` | Specify the type of hash that will be included in CEF messages. Supported values are: md5, sha1, sha256. | | configuration.cef.enableCefMsg | bool | `false` | Enable or disable sending CEF messages to syslog. Defaults to `false` to avoid flooding. | | configuration.classify | object | `-` | Configure settings for Worker analysis and classification of files using the Spectra Core static analysis engine. | | configuration.classify.certificates | bool | `true` | Enable checking whether file certificate passes the certificate validation, in addition to checking certificate whitelists and blacklists. | | configuration.classify.documents | bool | `true` | Enable document format threat detection. | | configuration.classify.emails | bool | `true` | Enable detection of phishing and other email threats. | | configuration.classify.hyperlinks | bool | `true` | Enable embedded hyperlinks detection. | | configuration.classify.ignoreAdware | bool | `false` | When set to true, classification results that match adware will be ignored. | | configuration.classify.ignoreHacktool | bool | `false` | When set to true, classification results that match hacktool will be ignored. | | configuration.classify.ignorePacker | bool | `false` | When set to true, classification results that match packer will be ignored. | | configuration.classify.ignoreProtestware | bool | `false` | When set to true, classification results that match protestware will be ignored. | | configuration.classify.ignoreRiskware | bool | `false` | When set to true, classification results that match riskware will be ignored. | | configuration.classify.ignoreSpam | bool | `false` | When set to true, classification results that match spam will be ignored. | | configuration.classify.ignoreSpyware | bool | `false` | When set to true, classification results that match spyware will be ignored. | | configuration.classify.images | bool | `true` | When true, the heuristic image classifier for supported file formats is used. | | configuration.classify.modelsLinuxGeneral | string | `"malicious"` | This setting controls how the ML Model affects the classification. Possible values: malicious, suspicious, ignored, disabled. | | configuration.classify.modelsScriptsAutoit | string | `"malicious"` | This setting controls how the ML Model affects the classification Possible values: malicious, suspicious, ignored, disabled. | | configuration.classify.modelsScriptsExcel | string | `"malicious"` | This setting controls how the ML Model affects the classification. Possible values: malicious, suspicious, ignored, disabled. | | configuration.classify.modelsScriptsPowershell | string | `"malicious"` | This setting controls how the ML Model affects the classification. Possible values: malicious, suspicious, ignored, disabled. | | configuration.classify.modelsScriptsPython | string | `"malicious"` | This setting controls how the ML Model affects the classification. Possible values: malicious, suspicious, ignored, disabled. | | configuration.classify.modelsScriptsVisualbasic | string | `"malicious"` | This setting controls how the ML Model affects the classification. Possible values: malicious, suspicious, ignored, disabled. | | configuration.classify.modelsWindowsBackdoor | string | `"malicious"` | This setting controls how the ML Model affects the classification. Possible values: malicious, suspicious, ignored, disabled. | | configuration.classify.modelsWindowsDownloader | string | `"malicious"` | This setting controls how the ML Model affects the classification. Possible values: malicious, suspicious, ignored, disabled. | | configuration.classify.modelsWindowsGeneral | string | `"malicious"` | This setting controls how the ML Model affects the classification. Possible values: malicious, suspicious, ignored, disabled. | | configuration.classify.modelsWindowsInfostealer | string | `"malicious"` | This setting controls how the ML Model affects the classification. Possible values: malicious, suspicious, ignored, disabled. | | configuration.classify.modelsWindowsKeylogger | string | `"malicious"` | This setting controls how the ML Model affects the classification. Possible values: malicious, suspicious, ignored, disabled. | | configuration.classify.modelsWindowsRansomware | string | `"malicious"` | This setting controls how the ML Model affects the classification. Possible values: malicious, suspicious, ignored, disabled. | | configuration.classify.modelsWindowsRiskware | string | `"malicious"` | This setting controls how the ML Model affects the classification. Possible values: malicious, suspicious, ignored, disabled. | | configuration.classify.modelsWindowsWorm | string | `"malicious"` | This setting controls how the ML Model affects the classification. Possible values: malicious, suspicious, ignored, disabled. | | configuration.classify.pecoff | bool | `true` | When true, the heuristic Windows executable classifier for supported PE file formats is used. | | configuration.cleanup | object | `-` | Configures how often the Worker file system is cleaned up. | | configuration.cleanup.fileAgeLimit | int | `1440` | Time before an unprocessed file present on the appliance is deleted, in minutes. | | configuration.cleanup.taskAgeLimit | int | `90` | Time before analysis reports and records of processed tasks are deleted, in minutes. | | configuration.cleanup.taskUnprocessedLimit | int | `1440` | Time before an incomplete processing task is canceled, in minutes. | | configuration.cloud | object | `-` | Configures integration with the Spectra Intelligence service or a T1000 instance to receive additional classification information. | | configuration.cloud.enabled | bool | `false` | Enable/disable connection. | | configuration.cloud.proxy | object | `-` | Configure an optional proxy connection. | | configuration.cloud.proxy.enabled | bool | `false` | Enable/disable proxy server. | | configuration.cloud.proxy.port | int | `8080` | Specify the TCP port number if using an HTTP proxy. Allowed range(s): 1 … 65535\. Required only if proxy is used. | | configuration.cloud.proxy.server | string | `""` | Proxy hostname or IP address for routing requests from the appliance to Spectra Intelligence. Required only if proxy is used. | | configuration.cloud.server | string | `"https://appliance-api.reversinglabs.com"` | Hostname or IP address of the Spectra Intelligence server. Required if Spectra Intelligence integration is enabled. Format: `https://`. | | configuration.cloud.timeout | int | `6` | Specify the number of seconds to wait when connecting to Spectra Intelligence before terminating the connection request. | | configuration.cloudAutomation | object | `-` | Configures the Worker to automatically submit files to Spectra Intelligence for antivirus scanning (in addition to local static analysis and remote reputation lookup (from previous antivirus scans)). | | configuration.cloudAutomation.dataChangeSubscribe | bool | `false` | Subscribe to the Spectra Intelligence data change notification mechanism. | | configuration.cloudAutomation.spexUpload | object | `-` | Scanning settings. | | configuration.cloudAutomation.spexUpload.enabled | bool | `false` | Enable/disable this feature. | | configuration.cloudAutomation.spexUpload.rescanEnabled | bool | `true` | Enable/disable rescan of files upon submission based on the configured interval to include the latest AV results in the reports. | | configuration.cloudAutomation.spexUpload.rescanThresholdInDays | int | `3` | Set the interval in days for triggering an AV rescan. If the last scan is older than the specified value, a rescan will be initiated. A value of 0 means files will be rescanned with each submission. | | configuration.cloudAutomation.spexUpload.scanUnpackedFiles | bool | `false` | Enable/disable sending unpacked files to Deep Cloud Analysis for scanning. Consumes roughly double the processing resources compared to standard analysis. | | configuration.cloudAutomation.waitForAvScansTimeoutInMinutes | int | `240` | Sets the maximum wait time (in minutes) for Deep Cloud Analysis to complete. If the timeout is reached, the report will be generated without the latest AV results. | | configuration.cloudAutomation.waitForAvScansToFinish | bool | `false` | If set to true, delays report generation until Deep Cloud Analysis completes, ensuring the latest AV results are included. | | configuration.cloudCache.cacheMaxSizePercentage | float | `6.25` | Maximum cache size expressed as a percentage of the total allocated RAM on the Worker. Allowed range: 5 \- 15\. | | configuration.cloudCache.cleanupWindow | int | `10` | How often to run the cache cleanup process, in minutes. It is advisable for this value to be lower, or at least equal to the TTL value. Allowed range: 5 \- 60\. | | configuration.cloudCache.enabled | bool | `true` | Enable or disable the caching feature. | | configuration.cloudCache.maxIdleUpstreamConnections | int | `50` | The maximum number of idle upstream connections. Allowed range: 10 \- 50\. | | configuration.cloudCache.ttl | int | `240` | Time to live for cached records, in minutes. Allowed range: 1 \- 7200\. | | configuration.general.maxUploadSizeMb | int | `2048` | The largest file (in MB) that Worker will accept and start processing. Ignored if Spectra Intelligence is connected and file upload limits are set there. | | configuration.general.tsWorkerCheckThresholdMins | int | `720` | How often the processing service will be checked for timeouts. If any issues are detected, the process will be restarted. | | configuration.general.uploadSizeLimitEnabled | bool | `false` | Whether or not the upload size filter is active. Ignored if Spectra Intelligence is connected and file upload limits are set there. | | configuration.hashes | object | `-` | Spectra Core calculates file hashes during analysis and includes them in the analysis report. The following options configure which additional hash types should be calculated and included in the Worker report. SHA1 and SHA256 are always included and therefore aren’t configurable. Selecting additional hash types (especially SHA384 and SHA512) may slow report generation. | | configuration.hashes.enableCrc32 | bool | `false` | Include CRC32 hashes in reports. | | configuration.hashes.enableMd5 | bool | `true` | Include MD5 hashes in reports. | | configuration.hashes.enableSha384 | bool | `false` | Include SHA384 hashes in reports. | | configuration.hashes.enableSha512 | bool | `false` | Include SHA512 hashes in reports. | | configuration.hashes.enableSsdeep | bool | `false` | Include SSDEEP hashes in reports. | | configuration.hashes.enableTlsh | bool | `false` | Include TLSH hashes in reports. | | configuration.health | object | `-` | Configures system health check configuration. | | configuration.health.diskHigh | int | `95` | Threshold for high disk usage. | | configuration.health.diskPath | string | `"/scratch"` | Empty string disables disk status check. | | configuration.health.enableDiskUsageCheck | bool | `false` | Enable/disable disk usage check. | | configuration.health.enabled | bool | `true` | Enable/disable system health check. | | configuration.health.queueHigh | int | `2000` | Number of files allowed in the queue. | | configuration.health.runtime | int | `10` | Specifies the number of seconds over which disk usage is sampled to assess the disk status. | | configuration.logging | object | `-` | Configures the severity above which events will be logged or sent to a remote syslog server. Severity can be: INFO, WARNING, or ERROR. | | configuration.logging.tiscaleLogLevel | string | `"INFO"` | Events below this level will not be presented on standard output. | | configuration.msGraph | object | `-` | Configures the Microsoft Cloud Storage file integration. | | configuration.msGraph.enabled | bool | `false` | Turns the Microsoft Cloud Storage file integration on or off. | | configuration.msGraph.folder | string | `""` | Folder where samples will be stored in Microsoft Cloud Storage. | | configuration.msGraphGeneral | object | `-` | Configures the general options for the Microsoft Cloud Storage integration. | | configuration.msGraphGeneral.customDomain | string | `""` | Application’s custom domain configured in the Azure portal. | | configuration.msGraphGeneral.siteHostname | string | `""` | Used only if `storageType` is set to SharePoint. This is the SharePoint hostname. | | configuration.msGraphGeneral.siteRelativePath | string | `""` | SharePoint Online site relative path. Only used when `storageType` is set to SharePoint. | | configuration.msGraphGeneral.storageType | string | `"onedrive"` | Specifies the storage type. Supported values are: onedrive or sharepoint. | | configuration.msGraphGeneral.username | string | `""` | Used only if `storageType` is set to OneDrive. Specifies which user’s drive will be used. | | configuration.processing | object | `-` | Configure the Worker file processing capabilities to improve performance and load balancing. | | configuration.processing.cacheEnabled | bool | `false` | Enable/disable caching. When enabled, Spectra Core can skip reprocessing the same files (duplicates) if uploaded consecutively in a short period. | | configuration.processing.cacheTimeToLive | int | `0` | If file processing caching is enabled, specify how long (in seconds) the analysis reports should be preserved in the cache before they expire. A value of 0 uses the default. Default: 600\. Maximum: 86400\. | | configuration.processing.depth | int | `0` | Specifies how "deep" a file is unpacked. By default, when set to 0, Workers will unpack files recursively until no more files can be unpacked. Setting a value greater than 0 limits the depth of recursion, which can speed up analyses but provide less detail. | | configuration.processing.largefileThreshold | int | `100` | If advanced mode is enabled, files larger than this threshold (in MB) will be processed individually, one by one. This parameter is ignored in standard mode. | | configuration.processing.mode | int | `2` | Configures the Worker processing mode to improve load balancing. Supported modes are standard (1) and advanced (2). | | configuration.processing.timeout | int | `28800` | Specifies how many seconds the Worker should wait for a file to process before terminating the task. Default: 28800\. Maximum: 259200\. | | configuration.propagation | object | `-` | Configure advanced classification propagation options supported by the Spectra Core static analysis engine. When Spectra Core classifies files, the classification of a child file can be applied to the parent file. | | configuration.propagation.enabled | bool | `true` | Enable/disable the classification propagation feature. When propagation is enabled, files can be classified based on the content extracted from them. This means that files containing a malicious or suspicious file will also be considered malicious or suspicious. | | configuration.propagation.goodwareOverridesEnabled | bool | `true` | Enable/disable goodware overrides. When enabled, any files extracted from a parent file and whitelisted by certificate, source or user override can no longer be classified as malicious or suspicious. This is an advanced goodware whitelisting technique that can be used to reduce the amount of false positive detections. | | configuration.propagation.goodwareOverridesFactor | int | `1` | When goodware overrides are enabled, this parameter must be configured to determine the factor to which overrides will be applied. Supported values are 0 to 5, where zero represents the best trust factor (highest confidence that a sample contains goodware). Overrides will apply to files with a trust factor equal to or lower than the value configured here. | | configuration.report | object | `-` | Configure the contents of the Spectra Detect file analysis report. | | configuration.report.firstReportOnly | bool | `false` | If disabled, the reports for samples with child files will include relationships for all descendant files. Enabling this setting will only include relationship metadata for the root parent file to reduce redundancy. | | configuration.report.includeStrings | bool | `false` | When enabled, strings are included in the file analysis report. Spectra Core can extract strings from binaries. This can be useful but may result in extensive metadata. To reduce noise, the types of included strings can be customized in the strings section. | | configuration.report.networkReputation | bool | `false` | If enabled, analysis reports include a top-level `network_reputation` object with reputation information for every extracted network resource. For this feature, Spectra Intelligence must be configured on the Worker, and the `ticore.processingMode` option must be set to "best". | | configuration.report.relationships | bool | `false` | Includes sample relationship metadata in the file analysis report. When enabled, the relationships section lists the hashes of files found within the given file. | | configuration.reportAdl | object | `-` | Settings to configure how reports saved to Azure Data Lake are formatted. | | configuration.reportAdl.archiveSplitReport | bool | `true` | Enable sending a single, smaller archive of split report files to ADL instead of each file. Relevant only when the 'Split report' option is used. | | configuration.reportAdl.container | string | `""` | Container where reports will be stored. Required when this feature is enabled. | | configuration.reportAdl.enabled | bool | `false` | Enable/disable storing file processing reports to ADL. | | configuration.reportAdl.filenameTimestampFormat | string | `""` | File naming pattern for the report itself. A timestamp is appended to the SHA1 hash of the file. The timestamp format must follow the strftime specification and be enclosed in quotation marks. If not specified, the ISO 8601 format is used. | | configuration.reportAdl.folder | string | `""` | Specify the name of a folder where analysis reports will be stored. If the folder name is not provided, files are stored into the root of the configured container. | | configuration.reportAdl.folderOption | string | `"date_based"` | Select the naming pattern that will be used when automatically creating subfolders for storing analysis reports. Supported options are: date\_based (YYYY/mm/dd/HH), datetime\_based (YYYY/mm/dd/HH/MM/SS), and sha1\_based (using the first 4 characters of the file hash). | | configuration.reportAdl.maliciousOnly | bool | `false` | When set, the report will only contain malicious and suspicious children. | | configuration.reportAdl.reportType | string | `"large"` | Specify the report type that should be applied to the Worker analysis report before storing it. Report types are results of filtering the full report. In other words, fields can be included or excluded as required. Report types are stored in the /etc/ts-report/report-types directory. | | configuration.reportAdl.splitReport | bool | `false` | By default, reports contain information on parent files and all extracted children files. When this option is enabled, analysis reports for extracted files are separated from their parent file report, and saved as individual report files. | | configuration.reportAdl.timestampEnabled | bool | `true` | Enable/disable appending a timestamp to the report name. | | configuration.reportAdl.topContainerOnly | bool | `false` | When enabled, the file analysis report will only include metadata for the top container and subreports for unpacked files will not be generated. | | configuration.reportAdl.view | string | `""` | Apply a view for transforming report data to the “large” report type to ensure maximum compatibility. Several existing views are also available as report types, which should be used as a view substitute due to performance gains. Custom views can be defined by placing the scripts in the “/usr/libexec/ts-report-views.d” directory on Spectra Detect Worker. | | configuration.reportApi | object | `-` | Configures the settings applied to the file analysis report fetched using the GET endpoint. To modify synchronous API timeouts or connection limits, apply the appropriate annotations to your Ingress resource. | | configuration.reportApi.maliciousOnly | bool | `false` | Report contains only malicious and suspicious children. | | configuration.reportApi.reportType | string | `"large"` | Specify the report type that should be applied to the Worker analysis report before storing it. Report types are results of filtering the full report. In other words, fields can be included or excluded as required. Report types are stored in the /etc/ts-report/report-types directory. | | configuration.reportApi.topContainerOnly | bool | `false` | When enabled, the file analysis report will only include metadata for the top container and subreports for unpacked files will not be generated. | | configuration.reportApi.view | string | `""` | Apply a view for transforming report data to the “large” report type to ensure maximum compatibility. Several existing views are also available as report types, which should be used as a view substitute due to performance gains. Custom views can be defined by placing the scripts in the “/usr/libexec/ts-report-views.d” directory on Spectra Detect Worker. | | configuration.reportMsGraph | object | `-` | Settings to configure how reports saved to OneDrive or SharePoint are formatted. | | configuration.reportMsGraph.archiveSplitReport | bool | `true` | Enable sending a single, smaller archive of split report files to Microsoft Cloud Storage instead of each file. Relevant only when the "Split Report" option is used. | | configuration.reportMsGraph.enabled | bool | `false` | Enable/disable storing file processing reports. | | configuration.reportMsGraph.filenameTimestampFormat | string | `""` | This refers to the naming of the report file itself. A timestamp is appended to the SHA1 hash of the file. The timestamp format must follow the strftime specification and be enclosed in quotation marks. If not specified, the ISO 8601 format is used. | | configuration.reportMsGraph.folder | string | `""` | Folder where report files will be stored on the Microsoft Cloud Storage. If the folder name is not provided, files are stored into the root of the configured container. | | configuration.reportMsGraph.folderOption | string | `"date_based"` | Select the naming pattern that will be used when automatically creating subfolders for storing analysis reports. Supported options are: date\_based (YYYY/mm/dd/HH), datetime\_based (YYYY/mm/dd/HH/MM/SS), and sha1\_based (using the first 4 characters of the file hash). | | configuration.reportMsGraph.maliciousOnly | bool | `false` | When set, the report will only contain malicious and suspicious children. | | configuration.reportMsGraph.reportType | string | `"large"` | Specify the report type that should be applied to the Worker analysis report before storing it. Report types are results of filtering the full report. In other words, fields can be included or excluded as required. Report types are stored in the /etc/ts-report/report-types directory. | | configuration.reportMsGraph.splitReport | bool | `false` | By default, reports contain information on parent files and all extracted children files. When this option is enabled, analysis reports for extracted files are separated from their parent file report, and saved as individual report files. | | configuration.reportMsGraph.topContainerOnly | bool | `false` | When enabled, file analysis report will only include metadata for the top container, and subreports for unpacked files will not be generated. | | configuration.reportMsGraph.view | string | `""` | Apply a view for transforming report data to the “large” report type to ensure maximum compatibility. Several existing views are also available as report types, which should be used as a view substitute due to performance gains. Custom views can be defined by placing the scripts in the “/usr/libexec/ts-report-views.d” directory on Spectra Detect Worker. | | configuration.reportS3 | object | `-` | Settings to configure how reports saved to S3 buckets are formatted. | | configuration.reportS3.advancedFilterEnabled | bool | `false` | Enable/disable usage of the advanced filter. | | configuration.reportS3.advancedFilterName | string | `""` | Name of the advanced filter. | | configuration.reportS3.archiveSplitReport | bool | `true` | Enable sending a single, smaller archive of split report files to S3 instead of each file. Relevant only when the 'Split report' option is used. | | configuration.reportS3.bucketMapping | object | `{}` | Used if `destinationType` is set to `mapping`. Accepts a dictionary of S3 input buckets mapped to output buckets, enclosed in quotation marks. | | configuration.reportS3.bucketName | string | `""` | Name of the S3 bucket where processed files will be stored. Required when this feature is enabled. | | configuration.reportS3.bucketS3ConnectionMapping | list | `[]` | List of structures that sets individual AWS connection methods for each target output bucket. | | configuration.reportS3.destinationType | string | `"default"` | Supported values are default (saves the reports into the bucket configured by bucketName), source (saves the reports into the S3 bucket where the samples originated from), mapping (saves the reports according to the mapping configured by bucketMapping). | | configuration.reportS3.enabled | bool | `false` | Enable/disable storing file processing reports to S3. | | configuration.reportS3.filenameTimestampFormat | string | `""` | This refers to the naming of the report file itself. A timestamp is appended to the SHA1 hash of the file. The timestamp format must follow the strftime specification and be enclosed in quotation marks. If not specified, the ISO 8601 format is used. | | configuration.reportS3.folder | string | `""` | Folder where report files will be stored in the given S3 bucket. The folder can be up to 1024 bytes long when encoded in UTF-8, and can contain letters, numbers and special characters: "\!", "-", "\_", ".", "\*", "'", "(", ")", "/". It must not start or end with a slash or contain leading or trailing spaces. Consecutive slashes ("//") are not allowed. | | configuration.reportS3.folderOption | string | `"date_based"` | Select the naming pattern used when automatically creating subfolders for storing analysis reports. Supported options are: date\_based (YYYY/mm/dd/HH), datetime\_based (YYYY/mm/dd/HH/MM/SS), and sha1\_based (using the first 4 characters of the file hash). | | configuration.reportS3.maliciousOnly | bool | `false` | When set, the report will only contain malicious and suspicious children. | | configuration.reportS3.reportType | string | `"large"` | Specify the report type that should be applied to the Worker analysis report before storing it. Report types are results of filtering the full report. In other words, fields can be included or excluded as required. Report types are stored in the /etc/ts-report/report-types directory. | | configuration.reportS3.splitReport | bool | `false` | By default, reports contain information on parent files and all extracted children files. When this option is enabled, analysis reports for extracted files are separated from their parent file report, and saved as individual report files. | | configuration.reportS3.timestampEnabled | bool | `true` | Enable/disable appending a timestamp to the report name. | | configuration.reportS3.topContainerOnly | bool | `false` | When enabled, the file analysis report will only include metadata for the top container and subreports for unpacked files will not be generated. | | configuration.reportS3.view | string | `""` | Apply a view for transforming report data to the “large” report type to ensure maximum compatibility. Several existing views are also available as report types, which should be used as a view substitute due to performance gains. Custom views can be defined by placing the scripts in the “/usr/libexec/ts-report-views.d” directory on Spectra Detect Worker. | | configuration.s3 | object | `-` | Settings for storing a copy of all files uploaded for analysis on Worker to an S3 or a third-party, S3-compatible server. | | configuration.s3.advancedFilterEnabled | bool | `false` | Enable/disable usage of the advanced filter. | | configuration.s3.advancedFilterName | string | `""` | Name of the advanced filter. | | configuration.s3.bucketMapping | object | `{}` | Used if `destinationType` is set to `mapping`. Accepts a dictionary of S3 input buckets mapped to output buckets, enclosed in quotation marks. | | configuration.s3.bucketName | string | `""` | Name of the S3 bucket where processed files will be stored. Required when this feature is enabled. | | configuration.s3.bucketS3ConnectionMapping | list | `[]` | List of structures that sets individual AWS connection methods for each target output bucket. | | configuration.s3.destinationType | string | `"default"` | Supported values are `default` (saves reports into the bucket configured by `bucketName`) and `mapping` (saves reports according to `bucketMapping`). | | configuration.s3.enabled | bool | `false` | Enable/disable storing file processed files on S3. | | configuration.s3.folder | string | `""` | Specify the name of a folder where analyzed files will be stored. If the folder name is not provided, files are stored into the root of the configured bucket. | | configuration.s3.storeMetadata | bool | `true` | When true, analysis metadata will be stored to the uploaded S3 object. | | configuration.scaling | object | `-` | Configures the number of concurrent processes and the number of files analyzed concurrently. Parameters in this section can be used to optimize the file processing performance on Worker. | | configuration.scaling.postprocessing | int | `1` | Specify how many post-processing instances to run. Post-processing instances will then modify and save reports or upload processed files to external storage. Increasing this value can increase throughput for servers with extra available cores. Maximum: 256\. | | configuration.scaling.preprocessingUnpacker | int | `1` | Specify how many copies of Spectra Core are used to unpack samples for Deep Cloud Analysis. This setting only has effect if Deep Cloud Analysis is enabled with Scan Unpacked Files capability. | | configuration.scaling.processing | int | `1` | Specify how many copies of Spectra Core engine instances to run. Each instance starts threads to process files. Maximum: 256\. | | configuration.sns | object | `-` | Configures settings for publishing notifications about file processing status and links the reports to an Amazon SNS (Simple Notification Service) topic. | | configuration.sns.enabled | bool | `false` | Enable/disable publishing notifications to Amazon SNS. | | configuration.sns.topic | string | `""` | Specify the SNS topic ARN that the notifications should be published to. Prerequisite: the AWS account in the AWS settings must be given permission to publish to this topic. Required when this feature is enabled. | | configuration.spectraAnalyzeIntegration | object | `-` | Configuration settings to upload processed samples to configured Spectra Analyze. | | configuration.spectraAnalyzeIntegration.address | string | `""` | Spectra Analyze address. Required when this feature is enabled. Has to be in the following format: `https://`. | | configuration.spectraAnalyzeIntegration.advancedFilterEnabled | bool | `true` | Enable/disable the advanced filter. | | configuration.spectraAnalyzeIntegration.advancedFilterName | string | `"default_filter"` | Name of the advanced filter. | | configuration.spectraAnalyzeIntegration.enabled | bool | `false` | Enable/disable integration with Spectra Analyze. | | configuration.splunk | object | `-` | Configures integration with Splunk, a logging server that can receive Spectra Detect file analysis reports. | | configuration.splunk.caPath | string | `""` | Path to the certificate. | | configuration.splunk.chunkSizeMb | int | `0` | The maximum size (MB) of a single request sent to Splunk. If an analysis report exceeds this size, it will be split into multiple parts. The report is split into its subreports (for child files). A request can contain one or multiple subreports, as long as its total size doesn’t exceed this limit. The report is never split by size alone \- instead, complete subreports are always preserved and sent to Splunk. Default: 0 (disabled) | | configuration.splunk.enabled | bool | `false` | Enable/disable Splunk integration. | | configuration.splunk.host | string | `""` | Specify the hostname or IP address of the Splunk server that should connect to the Worker appliance. | | configuration.splunk.https | bool | `true` | If set to true, HTTPS will be used for sending information to Splunk. If set to false, HTTP is used. | | configuration.splunk.port | int | `8088` | Specify the TCP port of the Splunk server’s HTTP Event Collector. | | configuration.splunk.reportType | string | `"large"` | Specifies which report\_type is returned. By default or when empty, only the medium (summary) report is provided in the callback response. Set to small, medium or large to view results of filtering the full report. | | configuration.splunk.sslVerify | bool | `false` | If HTTPS is enabled, setting this to true will enable certificate verification. | | configuration.splunk.timeout | int | `5` | Specify how many seconds to wait for a response from the Splunk server before the request fails. If the request fails, the report will not be uploaded to the Splunk server, and an error will be logged. The timeout value must be greater than or equal to 1, and not greater than 999\. | | configuration.splunk.topContainerOnly | bool | `false` | Specifies if Splunk should receive the report for the top (parent) file only. If set to true, no subreports will be sent. | | configuration.splunk.view | string | `""` | Specifies whether a custom Report View should be applied to the file analysis report and returned in the response. | | configuration.strings | object | `-` | Configure the output of strings extracted from files during Spectra Core static analysis. | | configuration.strings.enableStringExtraction | bool | `false` | If set to true, user-provided criteria for string extraction will be used. | | configuration.strings.maxLength | int | `32768` | Maximum number of characters in strings. | | configuration.strings.minLength | int | `4` | Minimum number of characters in strings. Strings shorter than this value are not extracted. | | configuration.strings.unicodePrintable | bool | `false` | Specify whether strings are Unicode printable or not. | | configuration.strings.utf16be | bool | `true` | Allow/disallow extracting UTF-16BE strings. | | configuration.strings.utf16le | bool | `true` | Allow/disallow extracting UTF-16LE strings. | | configuration.strings.utf32be | bool | `false` | Allow/disallow extracting UTF-32BE strings. | | configuration.strings.utf32le | bool | `false` | Allow/disallow extracting UTF-32LE strings. | | configuration.strings.utf8 | bool | `true` | Allow/disallow extracting UTF-8 strings. | | configuration.ticore | object | `-` | Configures cloud options supported by Spectra Core. Worker must be connected to Spectra Intelligence for these settings to take effect. | | configuration.ticore.maxDecompressionFactor | float | `1.0` | Decimal value between 0 and 999.9. If multiple decimals are given, it will be rounded to one decimal. Used to protect the user from intentional or unintentional archive bombs, terminating decompression if the size of unpacked content exceeds a set quota. | | configuration.ticore.mwpExtended | bool | `false` | Enable/disable information from antivirus engines in Spectra Intelligence. | | configuration.ticore.mwpGoodwareFactor | int | `2` | Determines when a file classified as KNOWN in Spectra Intelligence Cloud is classified as Goodware by Spectra Core. By default, all KNOWN cloud classifications are converted to Goodware. Supported values are 0 \- 5, where zero represents the best trust factor (highest confidence that a sample contains goodware). Lowering the value reduces the number of samples classified as goodware. Samples with a trust factor above the configured value are considered UNKNOWN. | | configuration.ticore.processingMode | string | `"best"` | Determines which file formats are unpacked by Spectra Core for detailed analysis. "best" fully processes all supported formats; "fast" processes a limited set. | | configuration.ticore.useXref | bool | `false` | Enabling XREF service will enrich analysis reports with cross-reference metadata like AV scanner results. | | configuration.unpackedAdl | object | `-` | Settings for storing extracted files in an Azure Data Lake container. | | configuration.unpackedAdl.archiveUnpacked | bool | `true` | Enable sending a single, smaller archive of unpacked files to ADL instead of each unpacked file. | | configuration.unpackedAdl.container | string | `""` | Specify the name of the Azure Data Lake container where extracted files will be saved. Required when this feature is enabled. | | configuration.unpackedAdl.enabled | bool | `false` | Enable/disable storing extracted files to ADL. | | configuration.unpackedAdl.folder | string | `""` | Specify the name of a folder in the configured Azure container where extracted files will be stored. If the folder name is not provided, files are stored into the root of the configured container. | | configuration.unpackedAdl.folderOption | string | `"date_based"` | Select the naming pattern that will be used when automatically creating subfolders for storing analyzed files. Supported options are: date\_based (YYYY/mm/dd/HH), datetime\_based (YYYY/mm/dd/HH/MM/SS), and sha1\_based (using the first 4 characters of the file hash). | | configuration.unpackedMsGraph | object | `-` | Settings for storing extracted files to Microsoft Cloud Storage. | | configuration.unpackedMsGraph.archiveUnpacked | bool | `true` | Enable sending a single, smaller archive of unpacked files to Microsoft Cloud Storage instead of each unpacked file. | | configuration.unpackedMsGraph.enabled | bool | `false` | Enable/disable storing extracted files. | | configuration.unpackedMsGraph.folder | string | `""` | Folder where unpacked files will be stored on the Microsoft Cloud Storage. If the folder name is not provided, files are stored into the root of the configured container. | | configuration.unpackedMsGraph.folderOption | string | `"date_based"` | Select the naming pattern that will be used when automatically creating subfolders for storing analyzed files. Supported options are: date\_based (YYYY/mm/dd/HH), datetime\_based (YYYY/mm/dd/HH/MM/SS), and sha1\_based (using the first 4 characters of the file hash). | | configuration.unpackedS3 | object | `-` | Settings for storing extracted files to S3 container. | | configuration.unpackedS3.advancedFilterEnabled | bool | `false` | Enable/disable the use of advanced filters. | | configuration.unpackedS3.advancedFilterName | string | `""` | Name of the advanced filter. | | configuration.unpackedS3.archiveUnpacked | bool | `true` | Enable sending a single, smaller archive of unpacked files to S3 instead of each unpacked file. | | configuration.unpackedS3.bucketName | string | `""` | Specify the name of the S3 container where extracted files will be saved. Required when this feature is enabled. | | configuration.unpackedS3.enabled | bool | `false` | Enable/disable storing extracted files in S3. | | configuration.unpackedS3.folder | string | `""` | The name of a folder in the configured S3 container where extracted files will be stored. If the folder name is not provided, files are stored into the root of the configured container. The folder can be up to 1024 bytes long when encoded in UTF-8, and can contain letters, numbers and special characters: "\!", "-", "\_", ".", "\*", "'", "(", ")", "/". It must not start or end with a slash or contain leading or trailing spaces. Consecutive slashes ("//") are not allowed. | | configuration.unpackedS3.folderOption | string | `"date_based"` | Select the naming pattern that will be used when automatically creating subfolders for storing analyzed files. Supported options are: date\_based (YYYY/mm/dd/HH), datetime\_based (YYYY/mm/dd/HH/MM/SS), and sha1\_based (using the first 4 characters of the file hash). | | configuration.wordlist | list | `-` | List of passwords for protected files. | ### Worker Component Configuration Values | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | advancedFilters | object | `{}` | Contains key-value pairs in which keys are filter names and values are the filter definitions. | | applyRestrictedPolicy | bool | `false` | Apply restricted policy to the pods and containers so they can run in a namespace with restricted policy enabled. If umbrella is used for deployment, this value is ignored, and global.applyRestrictedPolicy is used. | | auth.image | object | `-` | Configuration values of the image used for authentication. | | auth.image.pullPolicy | string | `"Always"` | Image pull policy. Options: Always, IfNotPresent, Never. | | auth.image.tag | string | `"8.0.0-35"` | Image tag. | | auth.resources | object | `-` | Resource requests and limits for the container. | | auth.resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | auth.resources.limits.cpu | string | `"4000m"` | CPU limit. Throttling occurs if the container exceeds this value. | | auth.resources.limits.memory | string | `"256Mi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | auth.resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | auth.resources.requests.cpu | string | `"500m"` | CPU request. | | auth.resources.requests.memory | string | `"128Mi"` | Memory request. | | authReverseProxy.image | object | `-` | Configuration values of the auth reverse proxy image. | | authReverseProxy.image.pullPolicy | string | `"Always"` | Image pull policy. Options: Always, IfNotPresent, Never. | | authReverseProxy.image.repository | string | `"nginx"` | Image repository. | | authReverseProxy.image.tag | string | `"stable"` | Image tag. | | authReverseProxy.resources | object | `-` | Resource requests and limits for the container. | | authReverseProxy.resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | authReverseProxy.resources.limits.cpu | string | `"2000m"` | CPU limit. Throttling occurs if the container exceeds this value. | | authReverseProxy.resources.limits.memory | string | `"512Mi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | authReverseProxy.resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | authReverseProxy.resources.requests.cpu | string | `"250m"` | CPU request. | | authReverseProxy.resources.requests.memory | string | `"128Mi"` | Memory request. | | checkHealth.failedJobsHistoryLimit | int | `1` | Number of failed finished jobs to keep. | | checkHealth.image | object | `-` | Configuration values of the image used for health check job. | | checkHealth.image.pullPolicy | string | `"Always"` | Image pull policy. Options: Always, IfNotPresent, Never. | | checkHealth.image.tag | string | `"6.0.0-14"` | Image tag. | | checkHealth.resources | object | `-` | Resource requests and limits for the container. | | checkHealth.resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | checkHealth.resources.limits.cpu | string | `"2000m"` | CPU limit. Throttling occurs if the container exceeds this value. | | checkHealth.resources.limits.memory | string | `"2Gi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | checkHealth.resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | checkHealth.resources.requests.cpu | string | `"1000m"` | CPU request. | | checkHealth.resources.requests.memory | string | `"1Gi"` | Memory request. | | checkHealth.startingDeadlineSeconds | int | `60` | Deadline (in seconds) for starting the Job, if that Job misses its scheduled time for any reason. After missing the deadline, the CronJob skips that instance of the Job. | | checkHealth.successfulJobsHistoryLimit | int | `1` | Number of successful finished jobs to keep. | | cleanup.failedJobsHistoryLimit | int | `1` | Number of failed finished jobs to keep. | | cleanup.image | object | `-` | Configuration values of the image used for cleanup job. | | cleanup.image.pullPolicy | string | `"Always"` | Image pull policy. Options: Always, IfNotPresent, Never. | | cleanup.image.tag | string | `"6.0.0-14"` | Image tag. | | cleanup.resources | object | `-` | Resource requests and limits for the container. | | cleanup.resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | cleanup.resources.limits.cpu | string | `"2000m"` | CPU limit. Throttling occurs if the container exceeds this value. | | cleanup.resources.limits.memory | string | `"2Gi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | cleanup.resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | cleanup.resources.requests.cpu | string | `"1000m"` | CPU request. | | cleanup.resources.requests.memory | string | `"1Gi"` | Memory request. | | cleanup.startingDeadlineSeconds | int | `180` | Deadline (in seconds) for starting the Job, if that Job misses its scheduled time for any reason. After missing the deadline, the CronJob skips that instance of the Job. | | cleanup.successfulJobsHistoryLimit | int | `1` | Number of successful finished jobs to keep. | | cloudCache.image | object | `-` | Configuration values of the cloud cache image. | | cloudCache.image.pullPolicy | string | `"Always"` | Image pull policy. Options: Always, IfNotPresent, Never. | | cloudCache.image.tag | string | `"1.3.2-3"` | Image tag. | | cloudCache.resources | object | `-` | Resource requests and limits for the container. | | cloudCache.resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | cloudCache.resources.limits.cpu | string | `"4000m"` | CPU limit. Throttling occurs if the container exceeds this value. | | cloudCache.resources.limits.memory | string | `"4Gi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | cloudCache.resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | cloudCache.resources.requests.cpu | string | `"1000m"` | CPU request. | | cloudCache.resources.requests.memory | string | `"1Gi"` | Memory request. | | clusterDomainName | string | `"cluster.local"` | Cluster domain name. If umbrella is used for deployment, this value is ignored, and global.clusterDomainName is applied. | | configManager.enabled | bool | `false` | Whether to enable the config manager service. MUST be set to false if SDM Portal is not deployed or configManager will attempt to register with SDM indefinitely. | | configManager.image | object | `-` | Configuration values for the config manager service. | | configManager.image.pullPolicy | string | `"Always"` | Image pull policy. Options: Always, IfNotPresent, Never. | | configManager.image.tag | string | `"8.0.0-35"` | Image tag. | | configManager.resources | object | `-` | Resource requests and limits for the container. | | configManager.resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | configManager.resources.limits.cpu | string | `"1000m"` | CPU limit. Throttling occurs if the container exceeds this value. | | configManager.resources.limits.memory | string | `"256Mi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | configManager.resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | configManager.resources.requests.cpu | string | `"250m"` | CPU request. | | configManager.resources.requests.memory | string | `"128Mi"` | Memory request. | | configManager.useK8sApi | bool | `true` | Enables an additional endpoint to fetch configuration via the K8s API (requires RBAC permissions). | | imagePullSecrets | list | `["rl-registry-key"]` | Set of stored credentials (authentication tokens) that allows Kubernetes node to "log in" to a private container registry to pull restricted images. | | ingress | object | `-` | Ingress configuration values. | | ingress.annotations | object | `{}` | Custom annotations to fine-tune the Ingress Controller behavior. | | ingress.className | string | `"nginx"` | IngressClass that will handle this resource. Must match an existing 'IngressClass' in the cluster. | | ingress.enabled | bool | `true` | Enable/disable the creation of the Ingress resource. | | ingress.host | string | `""` | The Fully Qualified Domain Name (FQDN) for the application. Required if 'enabled' is true. | | ingress.tls | object | `-` | TLS / SSL Configuration. | | ingress.tls.certificateArn | string | `""` | The Amazon Resource Name (ARN) of the certificate for AWS Load Balancer Controller. Only applicable when 'className' is 'alb'. | | ingress.tls.issuer | string | `""` | The name of the cert-manager ClusterIssuer or Issuer to request certificates from. | | ingress.tls.issuerKind | string | `"Issuer"` | The resource type of the issuer. Typically, 'Issuer' (namespace-scoped) or 'ClusterIssuer' (cluster-wide). | | monitoring.enabled | bool | `false` | Enable/disable monitoring with Prometheus. | | monitoring.prometheusReleaseName | string | `"kube-prometheus-stack"` | Prometheus release name. | | persistence | object | `-` | Data storage configuration for storing samples and reports. | | persistence.accessModes | list | `["ReadWriteMany"]` | Specifies the access modes for the volume. When autoscaling or multiple worker is used should be set to `[ "ReadWriteMany" ]`. | | persistence.requestStorage | string | `"10Gi"` | The amount of storage to request. | | persistence.storageClassName | string | `nil` | Name of the StorageClass to use. When autoscaling or multiple worker is used storage class should support "ReadWriteMany". If EFS storage is used, disable disk health check by setting configuration.health.enableDiskUsageCheck to false. | | postgres.customSecretName | string | `nil` | Postgres custom secret name. If umbrella chart is used for deployment, this value is ignored, and the value from global.postgresCustomSecretName will be used as a secret name. | | postgres.releaseName | string | `""` | Postgres release name, required when deployment is not done with the umbrella chart. | | postprocessor.autoscaling | object | `-` | Autoscaling configuration values. | | postprocessor.autoscaling.cooldownPeriod | int | `180` | The period to wait after the last trigger reported active before scaling the resource back to 0, in seconds. | | postprocessor.autoscaling.enabled | bool | `true` | Enable/disable autoscaling. | | postprocessor.autoscaling.maxReplicas | int | `8` | Maximum number of replicas that can be deployed when scaling is enabled. | | postprocessor.autoscaling.minReplicas | int | `0` | Minimum number of replicas that need to be deployed. | | postprocessor.autoscaling.pollingInterval | int | `10` | Interval to check each trigger, in seconds. | | postprocessor.autoscaling.scaleDown | object | `-` | ScaleDown configuration values. | | postprocessor.autoscaling.scaleDown.stabilizationWindow | int | `180` | Number of continuous seconds in which the scaling condition is not met. When this is reached, scale down is started. | | postprocessor.autoscaling.scaleUp | object | `-` | ScaleUp configuration values. | | postprocessor.autoscaling.scaleUp.numberOfPods | int | `1` | Number of pods that can be scaled in the defined period. | | postprocessor.autoscaling.scaleUp.period | int | `30` | Interval in which the numberOfPods value is applied. | | postprocessor.autoscaling.scaleUp.stabilizationWindow | int | `15` | Number of continuous seconds in which the scaling condition is met. When this is reached, scale up is started. | | postprocessor.autoscaling.targetInputQueueSize | int | `10` | Number of messages in backlog to trigger scaling on. Must be greater than 0. | | postprocessor.image | object | `-` | Configuration values of the postprocessor image. | | postprocessor.image.pullPolicy | string | `"Always"` | Image pull policy. Options: Always, IfNotPresent, Never. | | postprocessor.image.tag | string | `"8.0.0-35"` | Image tag. | | postprocessor.replicaCount | int | `1` | Number of desired pod instances. Ignored when autoscaling is enabled. | | postprocessor.resources | object | `-` | Resource requests and limits for the container. | | postprocessor.resources.limits.cpu | string | `nil` | CPU limit. Throttling occurs if the container exceeds this value. | | postprocessor.resources.limits.memory | string | `"16Gi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | postprocessor.resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | postprocessor.resources.requests.cpu | string | `"2500m"` | CPU request. | | postprocessor.resources.requests.memory | string | `"2Gi"` | Memory request. | | preprocessor.autoscaling | object | `-` | Autoscaling configuration values. | | preprocessor.autoscaling.cooldownPeriod | int | `180` | The period to wait after the last trigger reported active before scaling the resource back to 0, in seconds. | | preprocessor.autoscaling.enabled | bool | `true` | Enable/disable autoscaling. | | preprocessor.autoscaling.maxReplicas | int | `8` | Maximum number of replicas that can be deployed when scaling is enabled. | | preprocessor.autoscaling.minReplicas | int | `0` | Minimum number of replicas that need to be deployed. | | preprocessor.autoscaling.pollingInterval | int | `10` | Interval to check each trigger, in seconds. | | preprocessor.autoscaling.scaleDown | object | `-` | ScaleDown configuration values. | | preprocessor.autoscaling.scaleDown.stabilizationWindow | int | `180` | Number of continuous seconds in which the scaling condition is not met. When this is reached, scale down is started. | | preprocessor.autoscaling.scaleUp | object | `-` | ScaleUp configuration values. | | preprocessor.autoscaling.scaleUp.numberOfPods | int | `1` | Number of pods that can be scaled in the defined period. | | preprocessor.autoscaling.scaleUp.period | int | `30` | Interval in which the numberOfPods value is applied. | | preprocessor.autoscaling.scaleUp.stabilizationWindow | int | `15` | Number of continuous seconds in which the scaling condition is met. When this is reached, scale up is started. | | preprocessor.autoscaling.targetInputQueueSize | int | `10` | Number of messages in backlog to trigger scaling on. Must be greater than 0. | | preprocessor.image | object | `-` | Configuration values of the preprocessor image. | | preprocessor.image.pullPolicy | string | `"Always"` | Image pull policy. Options: Always, IfNotPresent, Never. | | preprocessor.image.tag | string | `"8.0.0-35"` | Image tag. | | preprocessor.replicaCount | int | `1` | Number of desired pod instances. Ignored when autoscaling is enabled. | | preprocessor.resources | object | `-` | Resource requests and limits for the container. | | preprocessor.resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | preprocessor.resources.limits.cpu | string | `"4000m"` | CPU limit. Throttling occurs if the container exceeds this value. | | preprocessor.resources.limits.memory | string | `"4Gi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | preprocessor.resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | preprocessor.resources.requests.cpu | string | `"1000m"` | CPU request. | | preprocessor.resources.requests.memory | string | `"1Gi"` | Memory request. | | preprocessorUnpacker.autoscaling | object | `-` | Autoscaling configuration values. | | preprocessorUnpacker.autoscaling.cooldownPeriod | int | `180` | The period to wait after the last trigger reported active before scaling the resource back to 0, in seconds. | | preprocessorUnpacker.autoscaling.enabled | bool | `true` | Enable/disable autoscaling. | | preprocessorUnpacker.autoscaling.maxReplicas | int | `8` | Maximum number of replicas that can be deployed when scaling is enabled. | | preprocessorUnpacker.autoscaling.minReplicas | int | `0` | Minimum number of replicas that need to be deployed. | | preprocessorUnpacker.autoscaling.pollingInterval | int | `10` | Interval to check each trigger, in seconds. | | preprocessorUnpacker.autoscaling.scaleDown | object | `-` | ScaleDown configuration values. | | preprocessorUnpacker.autoscaling.scaleDown.stabilizationWindow | int | `180` | Number of continuous seconds in which the scaling condition is not met. When this is reached, scale down is started. | | preprocessorUnpacker.autoscaling.scaleUp | object | `-` | ScaleUp configuration values. | | preprocessorUnpacker.autoscaling.scaleUp.numberOfPods | int | `1` | Number of pods that can be scaled in the defined period. | | preprocessorUnpacker.autoscaling.scaleUp.period | int | `30` | Interval in which the numberOfPods value is applied. | | preprocessorUnpacker.autoscaling.scaleUp.stabilizationWindow | int | `15` | Number of continuous seconds in which the scaling condition is met. When this is reached, scale up is started. | | preprocessorUnpacker.autoscaling.targetInputQueueSize | int | `10` | Number of messages in backlog to trigger scaling on. Must be greater than 0. | | preprocessorUnpacker.replicaCount | int | `1` | Number of desired pod instances. Ignored when autoscaling is enabled. | | preprocessorUnpacker.resources | object | `-` | Resource requests and limits for the container. | | preprocessorUnpacker.resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | preprocessorUnpacker.resources.limits.cpu | string | `nil` | CPU limit. Throttling occurs if the container exceeds this value. | | preprocessorUnpacker.resources.limits.memory | string | `"16Gi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | preprocessorUnpacker.resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | preprocessorUnpacker.resources.requests.cpu | string | `"4000m"` | CPU request. | | preprocessorUnpacker.resources.requests.memory | string | `"4Gi"` | Memory request. | | preprocessorUnpacker.scaling.concurrencyCount | int | `0` | Defines the number of concurrent threads per Spectra Detect instance that should be used for processing. When set to 0, number of threads equals to the number of CPU cores on the system. Modifying this option may impact system performance. Consult with ReversingLabs Support before making any changes to the parameter. | | preprocessorUnpacker.scaling.prefetchCount | int | `4` | Defines the maximum number of individual files that can simultaneously be processed by a single instance of Spectra Core. When one file is processed, another from the queue enters the processing state. Must be greater than 0. | | processor.autoscaling | object | `-` | Autoscaling configuration values. | | processor.autoscaling.cooldownPeriod | int | `180` | The period to wait after the last trigger reported active before scaling the resource back to 0, in seconds. | | processor.autoscaling.enabled | bool | `true` | Enable/disable autoscaling. | | processor.autoscaling.maxReplicas | int | `8` | Maximum number of replicas that can be deployed when scaling is enabled. | | processor.autoscaling.minReplicas | int | `0` | Minimum number of replicas that need to be deployed. | | processor.autoscaling.pollingInterval | int | `10` | Interval to check each trigger, in seconds. | | processor.autoscaling.scaleDown | object | `-` | ScaleDown configuration values. | | processor.autoscaling.scaleDown.stabilizationWindow | int | `180` | Number of continuous seconds in which the scaling condition is not met. When this is reached, scale down is started. | | processor.autoscaling.scaleUp | object | `-` | ScaleUp configuration values. | | processor.autoscaling.scaleUp.numberOfPods | int | `1` | Number of pods that can be scaled in the defined period. | | processor.autoscaling.scaleUp.period | int | `30` | Interval in which the numberOfPods value is applied. | | processor.autoscaling.scaleUp.stabilizationWindow | int | `15` | Number of continuous seconds in which the scaling condition is met. When this is reached, scale up is started. | | processor.autoscaling.targetInputQueueSize | int | `10` | Number of messages in backlog to trigger scaling on. Must be greater than 0. | | processor.image | object | `-` | Configuration values of the processor image. | | processor.image.pullPolicy | string | `"Always"` | Image pull policy. Options: Always, IfNotPresent, Never. | | processor.image.tag | string | `"8.0.0-35"` | Image tag. | | processor.replicaCount | int | `1` | Number of desired pod instances. Ignored when autoscaling is enabled. | | processor.resources | object | `-` | Resource requests and limits for the container. | | processor.resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | processor.resources.limits.cpu | string | `nil` | CPU limit. Throttling occurs if the container exceeds this value. | | processor.resources.limits.memory | string | `"32Gi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | processor.resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | processor.resources.requests.cpu | string | `"4000m"` | CPU request. | | processor.resources.requests.memory | string | `"4Gi"` | Memory request. | | processor.scaling.concurrencyCount | int | `0` | Defines the number of concurrent threads per Spectra Detect instance that should be used for processing. When set to 0, number of threads equals to the number of CPU cores on the system. Modifying this option may impact system performance. Consult with ReversingLabs Support before making any changes to the parameter. | | processor.scaling.prefetchCount | int | `8` | Defines the maximum number of individual files that can simultaneously be processed by a single instance of Spectra Core. When one file is processed, another from the queue enters the processing state. Must be greater than 0. | | processorRetry.autoscaling | object | `-` | Autoscaling configuration values. | | processorRetry.autoscaling.cooldownPeriod | int | `180` | The period to wait after the last trigger reported active before scaling the resource back to 0, in seconds. | | processorRetry.autoscaling.enabled | bool | `true` | Enable/disable autoscaling. | | processorRetry.autoscaling.maxReplicas | int | `8` | Maximum number of replicas that can be deployed when scaling is enabled. | | processorRetry.autoscaling.minReplicas | int | `0` | Minimum number of replicas that need to be deployed. | | processorRetry.autoscaling.pollingInterval | int | `10` | Interval to check each trigger, in seconds. | | processorRetry.autoscaling.scaleDown | object | `-` | ScaleDown configuration values. | | processorRetry.autoscaling.scaleDown.stabilizationWindow | int | `180` | Number of continuous seconds in which the scaling condition is not met. When this is reached, scale down is started. | | processorRetry.autoscaling.scaleUp | object | `-` | ScaleUp configuration values. | | processorRetry.autoscaling.scaleUp.numberOfPods | int | `1` | Number of pods that can be scaled in the defined period. | | processorRetry.autoscaling.scaleUp.period | int | `30` | Interval in which the numberOfPods value is applied. | | processorRetry.autoscaling.scaleUp.stabilizationWindow | int | `15` | Number of continuous seconds in which the scaling condition is met. When this is reached, scale up is started. | | processorRetry.autoscaling.targetInputQueueSize | int | `10` | Number of messages in backlog to trigger scaling on. Must be greater than 0. | | processorRetry.replicaCount | int | `1` | Number of desired pod instances. Ignored when autoscaling is enabled. | | processorRetry.resources | object | `-` | Resource requests and limits for the container. | | processorRetry.resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | processorRetry.resources.limits.cpu | string | `nil` | CPU limit. Throttling occurs if the container exceeds this value. | | processorRetry.resources.limits.memory | string | `"64Gi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | processorRetry.resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | processorRetry.resources.requests.cpu | string | `"4000m"` | CPU request. | | processorRetry.resources.requests.memory | string | `"8Gi"` | Memory request. | | processorRetry.scaling.concurrencyCount | int | `0` | Defines the number of concurrent threads per Spectra Detect instance that should be used for processing. When set to 0, number of threads equals to the number of CPU cores on the system. Modifying this option may impact system performance. Consult with ReversingLabs Support before making any changes to the parameter. | | processorRetry.scaling.prefetchCount | int | `1` | Defines the maximum number of individual files that can simultaneously be processed by a single instance of Spectra Core. When one file is processed, another from the queue enters the processing state. Must be greater than 0. Recommended value for this type of processor is 1. | | rabbitmq.customAdminSecretName | string | `nil` | RabbitMQ custom admin secret name. If umbrella chart is used for deployment, this value is ignored, and the value from global.rabbitmqAdminCustomSecretName will be used as a secret name. | | rabbitmq.customSecretName | string | `nil` | RabbitMQ custom secret name. If umbrella chart is used for deployment, this value is ignored, and the value from global.rabbitmqCustomSecretName will be used as a secret name. | | rabbitmq.releaseName | string | `""` | Rabbitmq release name, required when deployment is not done with umbrella. | | receiver.autoscaling | object | `-` | Autoscaling configuration values. | | receiver.autoscaling.cooldownPeriod | int | `180` | The period to wait after the last trigger reported active before scaling the resource back to 0, in seconds. | | receiver.autoscaling.enabled | bool | `true` | Enable/disable autoscaling. | | receiver.autoscaling.maxReplicas | int | `8` | Maximum number of replicas that can be deployed when scaling is enabled. | | receiver.autoscaling.minReplicas | int | `1` | Minimum number of replicas that need to be deployed. | | receiver.autoscaling.pollingInterval | int | `10` | Interval to check each trigger, in seconds. | | receiver.autoscaling.scaleDown | object | `-` | ScaleDown configuration values. | | receiver.autoscaling.scaleDown.stabilizationWindow | int | `180` | Number of continuous seconds in which the scaling condition is not met. When this is reached, scale down is started. | | receiver.autoscaling.scaleUp | object | `-` | ScaleUp configuration values. | | receiver.autoscaling.scaleUp.numberOfPods | int | `1` | Number of pods that can be scaled in the defined period. | | receiver.autoscaling.scaleUp.period | int | `30` | Interval in which the numberOfPods value is applied. | | receiver.autoscaling.scaleUp.stabilizationWindow | int | `30` | Number of continuous seconds in which the scaling condition is met. When this is reached, scale up is started. | | receiver.autoscaling.triggerCPUValue | int | `75` | CPU value (in percentage), which will cause scaling when reached. The percentage is taken from the resource.limits.cpu value. Limits have to be set up. | | receiver.image | object | `-` | Configuration values of the receiver image. | | receiver.image.pullPolicy | string | `"Always"` | Image pull policy. Options: Always, IfNotPresent, Never. | | receiver.image.tag | string | `"8.0.0-35"` | Image tag. | | receiver.initImage | object | `-` | Configuration values of the image used for receiver initialization. | | receiver.initImage.pullPolicy | string | `"Always"` | Image pull policy. Options: Always, IfNotPresent, Never. | | receiver.initImage.tag | string | `"6.0.0-14"` | Image tag. | | receiver.replicaCount | int | `1` | Number of desired pod instances. Ignored when autoscaling is enabled. | | receiver.resources | object | `-` | Resource requests and limits for the container. | | receiver.resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | receiver.resources.limits.cpu | string | `"5000m"` | CPU limit. Throttling occurs if the container exceeds this value. | | receiver.resources.limits.memory | string | `"8Gi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | receiver.resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | receiver.resources.requests.cpu | string | `"2000m"` | CPU request. | | receiver.resources.requests.memory | string | `"1Gi"` | Memory request. | | report.autoscaling | object | `-` | Autoscaling configuration values. | | report.autoscaling.cooldownPeriod | int | `180` | The period to wait after the last trigger reported active before scaling the resource back to 0, in seconds. | | report.autoscaling.enabled | bool | `true` | Enable/disable autoscaling. | | report.autoscaling.maxReplicas | int | `8` | Maximum number of replicas that can be deployed when scaling in enabled. | | report.autoscaling.minReplicas | int | `1` | Minimum number of replicas that need to be deployed. | | report.autoscaling.pollingInterval | int | `10` | Interval to check each trigger, in seconds. | | report.autoscaling.scaleDown | object | `-` | ScaleDown configuration values. | | report.autoscaling.scaleDown.stabilizationWindow | int | `180` | Number of continuous seconds in which the scaling condition is not met. When this is reached, scale down is started. | | report.autoscaling.scaleUp | object | `-` | ScaleUp configuration values. | | report.autoscaling.scaleUp.numberOfPods | int | `1` | Number of pods that can be scaled in the defined period. | | report.autoscaling.scaleUp.period | int | `30` | Interval in which the numberOfPods value is applied. | | report.autoscaling.scaleUp.stabilizationWindow | int | `30` | Number of continuous seconds in which the scaling condition is met. When this is reached, scale up is started. | | report.autoscaling.triggerCPUValue | int | `75` | CPU value (in percentage), which will cause scaling when reached. The percentage is taken from the resource.limits.cpu value. Limits have to be set up. | | report.image | object | `-` | Configuration values of the report image. | | report.image.pullPolicy | string | `"Always"` | Image pull policy. Options: Always, IfNotPresent, Never. | | report.image.tag | string | `"8.0.0-35"` | Image tag. | | report.replicaCount | int | `1` | Number of desired pod instances. Ignored when autoscaling is enabled. | | report.resources | object | `-` | Resource requests and limits for the container. | | report.resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | report.resources.limits.cpu | string | `"8000m"` | CPU limit. Throttling occurs if the container exceeds this value. | | report.resources.limits.memory | string | `"8Gi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | report.resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | report.resources.requests.cpu | string | `"2000m"` | CPU request. | | report.resources.requests.memory | string | `"2Gi"` | Memory request. | | reportTypes | object | `{}` | Contains key-value pairs where keys are the report type names and values are the report type definitions. | | sdmPortal | object | `-` | Reference to the SDM portal service for config manager. Required if SDM Portal is deployed. | | sdmPortal.namespace | string | `""` | The namespace where SDM Portal is deployed. Defaults to the Worker's namespace if left empty. | | sdmPortal.port | string | `nil` | The port used to access the SDM Portal. Defaults to 8080 if not specified. | | sdmPortal.releaseName | string | `""` | The release name of the SDM Portal. Fill this out if centralManager.queueLoggingEnabled is true and SDM is deployed without an umbrella chart. | | sdmPortal.urlOverride | string | `""` | Full URL of the SDM Portal (e.g., http://sdm.example.com). If provided, releaseName and namespace are ignored. Use this if SDM is outside the cluster. | | tcScratch | object | `-` | tcScratch values configure generic ephemeral volume options for the Spectra Core `/tc-scratch` directory. | | tcScratch.accessModes | list | `["ReadWriteOnce"]` | Access modes. | | tcScratch.requestStorage | string | `"100Gi"` | Requested storage size for the ephemeral volume. | | tcScratch.storageClassName | string | `nil` | Sets the storage class for the ephemeral volume. If not set, `emptyDir` is used instead of an ephemeral volume. | | tclibs.autoscaling | object | `-` | Autoscaling configuration values. | | tclibs.autoscaling.cooldownPeriod | int | `180` | The period to wait after the last trigger reported active before scaling the resource back to 0, in seconds. | | tclibs.autoscaling.enabled | bool | `true` | Enable/disable autoscaling. | | tclibs.autoscaling.maxReplicas | int | `8` | Maximum number of replicas that can be deployed when scaling is enabled. | | tclibs.autoscaling.minReplicas | int | `1` | Minimum number of replicas that need to be deployed. | | tclibs.autoscaling.pollingInterval | int | `10` | Interval to check each trigger, in seconds. | | tclibs.autoscaling.scaleDown | object | `-` | ScaleDown configuration values. | | tclibs.autoscaling.scaleDown.stabilizationWindow | int | `180` | Number of continuous seconds in which the scaling condition is not met. When this is reached, scale down is started. | | tclibs.autoscaling.scaleUp | object | `-` | ScaleUp configuration values. | | tclibs.autoscaling.scaleUp.numberOfPods | int | `1` | Number of pods that can be scaled in the defined period. | | tclibs.autoscaling.scaleUp.period | int | `30` | Interval in which the numberOfPods value is applied. | | tclibs.autoscaling.scaleUp.stabilizationWindow | int | `30` | Number of continuous seconds in which the scaling condition is met. When this is reached, scale up is started. | | tclibs.autoscaling.triggerCPUValue | int | `75` | CPU value (in percentage), which will cause scaling when reached. The percentage is taken from the resource.limits.cpu value. Limits have to be set up. | | tclibs.image | object | `-` | Configuration values of the tcLibs image. | | tclibs.image.pullPolicy | string | `"Always"` | Image pull policy. Options: Always, IfNotPresent, Never. | | tclibs.image.tag | string | `"0.0.10-2"` | Image tag. | | tclibs.replicaCount | int | `1` | Number of desired pod instances. Ignored when autoscaling is enabled. | | tclibs.resources | object | `-` | Resource requests and limits for the container. | | tclibs.resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | tclibs.resources.limits.cpu | string | `"2000m"` | CPU limit. Throttling occurs if the container exceeds this value. | | tclibs.resources.limits.memory | string | `"2Gi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | tclibs.resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | tclibs.resources.requests.cpu | string | `"1000m"` | CPU request. | | tclibs.resources.requests.memory | string | `"1Gi"` | Memory request. | | yaraRules | object | `{}` | Object in which keys are names of yara rule files and values are the yara rule definitions. | | yaraSync.enabled | bool | `false` | Enables/disables yara sync. | | yaraSync.image | object | `-` | Configuration values of the yara sync image. | | yaraSync.image.pullPolicy | string | `"Always"` | Image pull policy. Options: Always, IfNotPresent, Never. | | yaraSync.image.tag | string | `"8.0.0-35"` | Image tag. | | yaraSync.resources | object | `-` | Resource requests and limits for the container. | | yaraSync.resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | yaraSync.resources.limits.cpu | string | `"2000m"` | CPU limit. Throttling occurs if the container exceeds this value. | | yaraSync.resources.limits.memory | string | `"2Gi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | yaraSync.resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | yaraSync.resources.requests.cpu | string | `"1000m"` | CPU request. | | yaraSync.resources.requests.memory | string | `"1Gi"` | Memory request. | ## Connector S3 ### Connector S3 Secrets | Secret (customSecretName is set) | Secret (fullNameOverride is set) | Secret (deployment with umbrella) | Secret (deployment without umbrella) | Type | Description | | :---- | :---- | :---- | :---- | :---- | :---- | | `` | `-secret-` | `-connector-s3-secret-` | `-secret-` | required | Authentication secret used to connect to AWS S3 or any S3-compatible storage system. | ### Connector S3 Application Configuration Values | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | configuration | object | `-` | Connector S3 configuration values. | | configuration.dbCleanupPollInterval | int | `7200` | Specifies time in seconds, in which the database cleanup will be run. | | configuration.dbCleanupSampleThresholdInDays | int | `21` | Number of previous days that the data will be preserved. | | configuration.diskHighPercent | int | `0` | Disk High Percent | | configuration.inputs | list | `[]` | Configuration for S3 File Storage Input. [S3 input](#connector-s3-application-configuration---file-storage-input-configuration). | | configuration.maxFileSize | int | `0` | The maximum sample size in bytes that will be transmitted from the connector to the appliance for analysis. Setting it to 0 will disable the option. | | configuration.maxUploadDelayTime | int | `10000` | Delay in milliseconds. In case the Worker cluster is under high load, this parameter is used to delay any new upload to the Worker cluster. The delay parameter will be multiplied by the internal factor determined by the load on the Worker cluster. | | configuration.maxUploadRetries | int | `100` | Number of times the connector will attempt to upload the file to the processing appliance. Upon reaching the number of retries it will be discarded. | | configuration.systemInfo | object | `-` | Configuration for S3 System Info. [S3 System Info](#connector-s3-application-configuration---system-info-values). | | configuration.uploadTimeout | int | `10000` | Period (in milliseconds) between upload attempts of the sample being re-uploaded. | | configuration.uploadTimeoutAlgorithm | string | `"exponential"` | The algorithm used for managing delays between re-uploading the samples into the processing appliance. In exponential, the delay is defined by multiplying the max upload timeout parameter by 2, until max value of 5 minutes. Linear backoff will always use the Max upload timeout value for the timeout period between re-uploads. Allowed values: "exponential", "linear". | ### Connector S3 Application Configuration - System Info Values | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | configuration.systemInfo.centralLogging | bool | `false` | Enable central logging. | | configuration.systemInfo.diskHighPercent | int | `0` | Dish high percent. | | configuration.systemInfo.fetchChannelSize | int | `40` | Fetch channel size. | | configuration.systemInfo.hostUUID | string | `""` | Host UUID. | | configuration.systemInfo.maxConnections | int | `10` | Max number of connections. | | configuration.systemInfo.maxSlowFetches | int | `12` | Max slow fetches. | | configuration.systemInfo.numberOfRetries | int | `300` | Number of retries. | | configuration.systemInfo.requestTimeout | int | `43200` | Timeout for requests. | | configuration.systemInfo.slowFetchChannelSize | int | `100` | Slow fetch channel size. | | configuration.systemInfo.slowFetchPause | int | `5` | Slow fetch pause. | | configuration.systemInfo.type | string | `"tiscale"` | Type. | | configuration.systemInfo.verifyCert | bool | `false` | Verify SSL certificate. | | configuration.systemInfo.version | string | `"5.6.0"` | Version. | | configuration.systemInfo.waitTimeout | int | `1000` | Wait timeout. | ### Connector S3 Application Configuration - File Storage Input Configuration | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | inputs\[n\] | list | `-` | Configmap Configuration for S3 File Storage Input. | | inputs\[n\].awsEnableArn | bool | `false` | Enable/disable the usage of AWS IAM roles to access S3 buckets without sharing secret keys. | | inputs\[n\].awsExternalRoleId | string | `""` | The external ID of the role that will be assumed. This can be any string. | | inputs\[n\].awsRoleArn | string | `""` | The role ARN created using the external role ID and an Amazon ID. In other words, the ARN which allows a Worker to obtain a temporary token, which then allows it to save to S3 buckets without a secret access key. | | inputs\[n\].awsRoleSessionName | string | `"ARNRoleSession"` | Name of the session visible in AWS logs. This can be any string. | | inputs\[n\].awsS3AccessKeyId | string | `""` | AWS S3 access key ID. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | inputs\[n\].awsS3SecretAccessKey | string | `""` | AWS S3 secret access key. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | inputs\[n\].bucket | string | `""` | Name of an existing S3 bucket which contains the samples to process. | | inputs\[n\].createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `awsS3AccessKeyId` and `awsS3SecretAccessKey` parameters. WARNING: Use this for convenience/testing only. | | inputs\[n\].customSecretName | string | `nil` | Name of the secret in which the S3 storage credentials can be found. If no value is set, default secret name will be used. | | inputs\[n\].deleteSourceFile | bool | `false` | Selecting the checkbox will allow the connector to delete source files on S3 storage after they have been processed. Required if 'require\_analyze' or 'post\_actions\_enabled' is true. | | inputs\[n\].endpoint | string | `""` | Custom S3 endpoint URL. Leave empty if using standard AWS S3. | | inputs\[n\].folder | string | `""` | The input folder inside the specified bucket which contains the samples to process. All other samples (except those classified as unknown) will be ignored. | | inputs\[n\].identifier | string | `""` | Unique name of S3 connection. Must contain only lowercase alphanumeric characters or hyphen (-). Must start and end with an alphanumeric character. Identifier length must be between 3 and 49 characters. | | inputs\[n\].knownBucket | string | `""` | Specify the bucket into which the connector will store files classified as 'Goodware'. If empty, the input bucket will be used. | | inputs\[n\].knownDestination | string | `"goodware"` | The folder into which the connector will store files classified as 'Goodware'. The folder is contained within the specified bucket field. | | inputs\[n\].maliciousBucket | string | `""` | Specify the bucket into which the connector will store files classified as 'Malicious'. If empty, the input bucket will be used. | | inputs\[n\].maliciousDestination | string | `"malware"` | The folder into which the connector will store files classified as 'Malicious'. The folder is contained within the specified bucket field. | | inputs\[n\].objectMetadataFilter.classification | list | `[]` | Classification. | | inputs\[n\].objectMetadataFilter.enabled | bool | `false` | Enable/disable selection criteria using metadata. | | inputs\[n\].objectMetadataFilter.threatName | list | `[]` | Threat name. | | inputs\[n\].paused | bool | `false` | Temporarily pause the continuous scanning of this Storage Input. This setting must be set to true to enable retro hunting. | | inputs\[n\].postActionsEnabled | bool | `false` | Disable/enable post actions for S3 connectors. | | inputs\[n\].priority | int | `5` | A higher Priority makes it more likely that files from this bucket will be processed first. The supported range is from 1 (highest) to 5 (lowest). Values outside of those minimum and maximum values will be replaced by the minimum or maximum, respectively. Multiple buckets may share the same priority. | | inputs\[n\].requireAnalyze | bool | `false` | Disable/enable the requirement for analysis of data processed by connector. | | inputs\[n\].serverSideEncryptionCustomerAlgorithm | string | `""` | Customer provided encryption algorithm. | | inputs\[n\].serverSideEncryptionCustomerKey | string | `""` | Customer provided encryption key. | | inputs\[n\].suspiciousBucket | string | `""` | Specify the bucket into which the connector will store files classified as 'Suspicious'. If empty, the input bucket will be used. | | inputs\[n\].suspiciousDestination | string | `"suspicious"` | The folder into which the connector will store files classified as 'Suspicious'. The folder is contained within the specified bucket field. | | inputs\[n\].unknownBucket | string | `""` | Specify the bucket into which the connector will store files classified as 'Unknown'. If empty, the input bucket will be used. | | inputs\[n\].unknownDestination | string | `"unknown"` | The folder into which the connector will store files classified as 'Unknown'. The folder is contained within the specified bucket field. | | inputs\[n\].verifySslCertificate | bool | `true` | Connect securely to the custom S3 instance. Deselect this to accept untrusted certificates. Applicable only when using a custom S3 endpoint. | | inputs\[n\].zone | string | `"us-east-1"` | AWS S3 region. | ### Connector S3 Component Configuration Values | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | applyRestrictedPolicy | bool | `false` | Apply restricted policy to the pods and containers so they can run in a namespace with restricted policy enabled. If umbrella is used for deployment, this value is ignored, and global.applyRestrictedPolicy is used. | | boltdb.claimName | string | `nil` | PVC name. If empty, default pvc name will be used. | | clusterDomainName | string | `"cluster.local"` | Cluster domain name. If umbrella is used for deployment, this value is ignored, and global.clusterDomainName is applied. | | enabled | bool | `false` | Enable or disable the S3 connector deployment. | | fullNameOverride | string | `""` | Overrides connector-s3 chart full name. | | image | object | `-` | Configuration values of the connector s3 image. | | image.imagePullPolicy | string | `"Always"` | Image pull policy. Options: Always, IfNotPresent, Never. | | image.tag | string | `"0.38.0-1"` | Image tag. | | imagePullSecrets | list | `["rl-registry-key"]` | Set of stored credentials (authentication tokens) that allows Kubernetes node to "log in" to a private container registry to pull restricted images. | | monitoring.enabled | bool | `false` | Enable/disable monitoring with Prometheus. | | monitoring.prometheusReleaseName | string | `"kube-prometheus-stack"` | Prometheus release name. | | nameOverride | string | `""` | Overrides connector-s3 chart name. | | persistence | object | `-` | Data storage configuration (BoltDB). | | persistence.accessModes | list | `["ReadWriteOnce"]` | Specifies the access modes for the volume. | | persistence.requestStorage | string | `"10Gi"` | The amount of storage to request. | | persistence.storageClassName | string | `nil` | Name of the StorageClass to use. | | receiver | object | `-` | Receiver configuration. Needed if connector is deployed as a standalone chart. | | receiver.baseUrl | string | `nil` | FQDN (Fully Qualified Domain Name) of the receiver service. Needed only if connector is deployed as a standalone chart. | | receiver.service | object | `-` | Receiver service configuration values. | | receiver.service.httpPort | int | `80` | HTTP port number the receiver service is listening on. | | resources | object | `-` | Resource requests and limits for the container. | | resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | resources.limits.cpu | string | `nil` | CPU limit. Throttling occurs if the container exceeds this value. | | resources.limits.memory | string | `"6Gi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | resources.requests.cpu | string | `"4000m"` | CPU request. | | resources.requests.memory | string | `"2Gi"` | Memory request. | | sharedStorage | object | `{"enabled":true,"mode":"moving"}` | Configuration for shared storage between connector and receiver/worker. | | sharedStorage.enabled | bool | `true` | Enables shared storage between connector and receiver/worker. | | sharedStorage.mode | string | `"moving"` | Defines if files are downloaded directly to the shared storage ('direct') or first to tmp and then moved ('moving'). Only applicable when sharedStorage is enabled. | | tmp | object | `-` | Configuration values for generic ephemeral volume for the connectors' `/data/connectors/connector-s3/tmp` directory. | | tmp.accessModes | list | `["ReadWriteOnce"]` | Specifies the access modes for the volume. | | tmp.requestStorage | string | `"100Gi"` | Requested storage size for the ephemeral volume. | | tmp.storageClassName | string | `nil` | Name of the StorageClass to use for the ephemeral volume. If not set, `emptyDir` is used instead of an ephemeral volume. | | worker.releaseName | string | `nil` | Spectra Detect Worker release for connector to connect to. It is required if the connector-s3 is deployed without umbrella chart. In case the worker it connects to was deployed with umbrella chart, the release name needs to be `-wrk`, otherwise it should be ``. | ## Spectra Detect Manager (SDM) | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | sdmPortal.config.centralFileStorage.enabled | bool | `false` | If enabled, the following SDM components will be deployed: SeaWeedFS | | sdmPortal.config.centralLogging.enabled | bool | `true` | If enabled, the following SDM components will be deployed: Data Change Service and Clickhouse. | | sdmPortal.enabled | bool | `true` | If enabled, the following SDM components will be deployed: Portal, Celery Worker, SeaweedFS and Postfix. | ### SDM Secrets Configuration Values | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | global.secrets.clickhouse | object | `-` | Secret configuration values for Clickhouse credentials. | | global.secrets.clickhouse.customSecretName | string | `""` | Custom secret name. If not set, default secret name will be used. | | global.secrets.ldap | object | `-` | LDAP secret configuration values | | global.secrets.ldap.cacert.customSecretName | string | `""` | Custom secret name of the secret containing PEM file with CA certificate under the `ca.pem` key. | | global.secrets.ldap.cert.customSecretName | string | `nil` | Custom secret name of the secret containing PEM encoded cert for client certificate authentication under the `cert.pem` key. | | global.secrets.ldap.credentials.customSecretName | string | `""` | Custom secret name which contains credentials used in LDAP authentication. If not set, default secret name will be used. | | global.secrets.oidc | object | `-` | Secret configuration values for setting the OIDC client credentials. | | global.secrets.oidc.customSecretName | string | `""` | Custom secret name. If not set, default secret name will be used. | | global.secrets.postfix | object | `-` | Secret configuration values for Postfix credentials. | | global.secrets.postfix.customSecretName | string | `""` | Custom secret name. If not set, default secret name will be used. | | global.secrets.saml | object | `-` | SAML secret configuration values | | global.secrets.saml.customSecretName | string | `""` | Custom secret name of the secret containing federation metadata under the `metadata.xml` key. | | global.secrets.tiCloud | object | `-` | Secret configuration values for setting the Spectra Intelligence credentials. | | global.secrets.tiCloud.customSecretName | string | `""` | Custom secret name. If not set, default secret name will be used. | | global.secrets.tiCloudProxy | object | `-` | Secret configuration values for setting the Spectra Intelligence proxy credentials. | | global.secrets.tiCloudProxy.customSecretName | string | `""` | Custom secret name. If not set, default secret name will be used. | ### SDM Portal #### SDM Portal Secrets | Secret (when custom secret name is used) | Secret (default secret name) | Type | Description | Used in Components (Pods) | | :---- | :---- | :---- | :---- | :---- | | `` | `-secret-sdm-cloud` | Required when related feature is enabled | Basic authentication secret which contains username and password for Spectra Intelligence authentication. Required when Spectra Intelligence is enabled (config.ticloud.enabled). Secret is either created manually (sdm-portal chart) or already exists. | Portal, Celery, DCS | | `` | `-secret-sdm-cloud-proxy` | Optional | Basic authentication secret which contains username and password for Spectra Intelligence proxy authentication. Secret is either created manually (sdm-portal chart) or already exists. | Portal, Celery | | `` | `-secret-sdm-oidc` | Required when related feature is enabled (config.oidc.enabled) | Opaque secret which contains rp_client_id and rp_client_secret for OIDC authentication. Required when OIDC authentication is enabled (config.oidc.enabled). Secret is either created manually (sdm-portal chart) or already exists. | Portal, Celery | | `` | `-secret-sdm-ldap` | Required when related feature is enabled | Opaque secret which contains bind_dn and bind_password for LDAP authentication. Required when LDAP authentication is enabled (config.ldap.enabled). Secret is either created manually (sdm-portal chart) or already exists. | Portal | | `` | `-secret-sdm-ldap-cacert` | Required when related feature is enabled | Opaque secret which contains CA certificate for LDAP authentication. Required when LDAP authentication is enabled, and TLS features are enabled (config.ldap.enabled and config.ldap.tls and config.ldap.tlsRequireCert). Secret is either created manually (sdm-portal chart) or already exists. | Portal | | `` | `-secret-sdm-ldap-cert` | Required when related feature is enabled | Opaque secret which contains TLS certificate for LDAP server. Required when LDAP authentication is enabled and TLS features are enabled (config.ldap.enabled and config.ldap.tls and config.ldap.tlsRequireCert). Secret is either created manually (sdm-portal chart) or already exists. | Portal | | `` | `-secret-sdm-saml-metadata` | Required when related feature is enabled | Opaque secret which contains SAML federation metadata. Required when SAML authentication is enabled (config.saml.enabled). Secret is either created manually (sdm-portal chart) or already exists. | Portal | #### SDM Portal Application Configuration Values | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | config | object | `-` | SDM Portal configuration values. | | config.centralConfig | object | `-` | Configure Central Configuration. | | config.centralConfig.enabled | boolean | `true` | Enable/disable central configuration feature. | | config.centralFileStorage | object | `-` | Configure file storage in Spectra Detect Manager, allowing connected Workers to store samples for pivoting to and reprocessing in Spectra Analyze. | | config.centralFileStorage.enabled | boolean | `false` | Enable/disable file storage. | | config.centralFileStorage.fileSizeLimit | int | `400` | File size limit in MiB. Samples larger than the set threshold will not be stored. Minimum: 1. Maximum: 400. | | config.centralFileStorage.ttl | int | `24` | Sample retention period in hours after which the uploaded samples will be removed from the Central File Storage. Minimum: 1. Maximum: 2160. | | config.centralLogging | object | `-` | Configure Central Logging, which is used to collect and display information about all events happening on connected Workers and Integrations. | | config.centralLogging.enabled | bool | `false` | Enable/disable central logging. | | config.centralLogging.retentionPeriod | int | `90` | Retention period in days. Minimum: 1. Maximum: 540. | | config.classificationChanges | object | `-` | Monitor classification changes from Spectra Intelligence directly on the Spectra Detect Dashboard. | | config.classificationChanges.enabled | bool | `false` | Subscribe to classification changes. | | config.deepCloudAnalysis | object | `-` | Configure Deep Cloud Analysis. This uploads files to Spectra Intelligence for scanning with multiple AV engines, refining the final verdict (classification), risk score, and threat name. Depending on the configuration, this may increase processing load and require additional resources. | | config.deepCloudAnalysis.enabled | bool | `false` | Enable/disable Deep Cloud Analysis. | | config.deepCloudAnalysis.scanner1 | string | `""` | AV scanner results to display in the Detections Overview preview. | | config.deepCloudAnalysis.scanner2 | string | `""` | AV scanner results to display in the Detections Overview preview. | | config.deepCloudAnalysis.scanner3 | string | `""` | AV scanner results to display in the Detections Overview preview. | | config.deepCloudAnalysis.scanner4 | string | `""` | AV scanner results to display in the Detections Overview preview. | | config.deepCloudAnalysis.scanner5 | string | `""` | AV scanner results to display in the Detections Overview preview. | | config.rlapp | object | `-` | General SDM Portal configuration. | | config.rlapp.allowedHosts | list | `[]` | A list of host/domain names that this application site can serve. | | config.rlapp.sessionCookieAge | int | `604800` | Duration of the login session, in seconds. Minimum: 60 (1 minute). Maximum: 7776000 (90 days). | | config.rlapp.sessionTimeoutAutomaticallyLogout | bool | `false` | If true, automatically log out inactive users. | | config.rlapp.sessionTimeoutPeriod | int | `600` | Period of inactivity before sign out, in seconds. Minimum: 60 (1 minute). Maximum: 2592000 (30 days). | | config.smtp | object | `-` | SMTP Settings. | | config.smtp.defaultFromEmail | string | `"online@reversinglabs.com"` | Default email address to use for automated correspondence. | | config.smtp.smtpUseTls | boolean | `false` | Use TLS. | | config.sync | object | `-` | Synchronization settings. | | config.sync.yaraRulesetsEnabled | boolean | `false` | Allow connected appliances to synchronize YARA rulesets. | | config.ticloud | object | `-` | Spectra Intelligence settings. | | config.ticloud.enabled | boolean | `false` | Enable Spectra Intelligence. | | config.ticloud.proxyHost | string | `nil` | Proxy hostname for routing requests from the appliance to Spectra Intelligence. | | config.ticloud.proxyPort | integer | `25` | Proxy port number. Minimum: 0. Maximum: 65535. | | config.ticloud.timeout | integer | `60` | Specifies how long to wait before the Spectra Intelligence connection times out. Minimum: 1. Maximum: 1000. | #### SDM Portal - LDAP Authentication Configuration | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | config.ldap | object | `-` | Settings for LDAP authentication. | | config.ldap.denyGroup | string | `nil` | Authentication will fail for any user that belongs to this group. | | config.ldap.enabled | boolean | `false` | Enable/disable LDAP authentication. | | config.ldap.groupSchemaClass | string | `"group"` | The `objectClass` value used when searching for groups. | | config.ldap.groupSchemaNameAttr | string | `"cn"` | The group name field. | | config.ldap.groupSchemaType | string | `"member"` | Group schema type. Allowed values: `uniqueMember` and `member`. | | config.ldap.groupSearchBaseDn | string | `nil` | Root node in LDAP from which to search for groups. Example: 'cn=users,dc=example,dc=com'. | | config.ldap.groupSearchScope | integer | `2` | Scope. Allowed values: 0 (Base), 1 (One level), 2 (Subtree), 3 (Subordinate). | | config.ldap.host | string | `""` | Hostname or IP of the server running LDAP. | | config.ldap.port | integer | `389` | LDAP server port. Minimum: 0. Maximum: 65535. | | config.ldap.requireGroup | string | `nil` | Authentication will fail for any user that does not belong to this group. Example: 'cn=enabled,ou=groups,dc=example,dc=com' | | config.ldap.tls | boolean | `true` | If true, use Transport Layer Security (TLS) connection. | | config.ldap.tlsRequireCert | boolean | `false` | If true, TLS certificate is required. | | config.ldap.userAttrMapEmail | string | `"mail"` | Field to map to the email address. | | config.ldap.userAttrMapFirstName | string | `"givenName"` | Field to map to the first name. | | config.ldap.userAttrMapLastName | string | `"sn"` | Field to map to the last name. | | config.ldap.userFlagsByGroupIsActive | string | `nil` | Users will be marked as active only if they belong to this group. Example: 'cn=active,ou=users,dc=example,dc=com'. | | config.ldap.userFlagsByGroupIsSuperuser | string | `nil` | Users will be marked as superusers only if they belong to this group. Example: 'cn=admins,ou=groups,dc=example,dc=com'. | | config.ldap.userSchemaClass | string | `"user"` | The `objectClass` value used when searching for users. | | config.ldap.userSchemaNameAttr | string | `"sAMAccountName"` | The username field. Examples: 'sAMAccountName' or 'cn'. | | config.ldap.userSearchBaseDn | string | `nil` | Root node in LDAP from which to search for users. Example: 'cn=users,dc=example,dc=com.' | | config.ldap.userSearchScope | integer | `2` | Scope. Allowed values: 0 (Base), 1 (One level), 2 (Subtree), 3 (Subordinate). | #### SDM Portal - OIDC Authentication Configuration | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | config.oidc | object | `-` | Configure authentication with an OpenID Connect client. | | config.oidc.accessDenyGroup | string | `nil` | Authentication will fail for any user that belongs to one or more of the provided group(s). Use the access groups delimiter to separate multiple group names. | | config.oidc.accessRequireGroup | string | `nil` | Authentication will fail for any user that does not belong to one or more of the provided group(s). Use the access groups delimiter to separate multiple group names. | | config.oidc.audience | string | `nil` | Identifies the intended recipient of the token. | | config.oidc.claimsSource | string | `"ID_TOKEN"` | Source used to extract user claims. Allowed values: `ID_TOKEN`, `USER_INFO_ENDPOINT`, `ACCESS_TOKEN`. | | config.oidc.clientType | string | `"CONFIDENTIAL"` | Authenticate with client secret (`CONFIDENTIAL`) or without (`PUBLIC`). | | config.oidc.enabled | boolean | `false` | Enable/disable authentication with OIDC. | | config.oidc.issuer | string | `nil` | Issuer. | | config.oidc.mapClaimAccessGroupsDelimiter | string | `nil` | Character to split the User Access Groups on. Optional. The maximum length of the delimiter is 2 characters. These characters must not be used in any access group name. | | config.oidc.mapClaimEmail | string | `"email"` | The claim containing the unique email address. | | config.oidc.mapClaimFirstName | string | `"given_name"` | The claim containing the user's first name. | | config.oidc.mapClaimGroups | string | `"group"` | The claim containing the list of user's groups. | | config.oidc.mapClaimGroupsDelimiter | string | `nil` | Character to split the Groups string on. Optional. | | config.oidc.mapClaimLastName | string | `"family_name"` | The claim containing the user's last name. | | config.oidc.mapClaimUsername | string | `"unique_name"` | The claim containing the unique username. | | config.oidc.opAuthorizationEndpoint | string | `nil` | URL of your OpenID Connect provider authorization endpoint. | | config.oidc.opJwksEndpoint | string | `nil` | URL of your OpenID Connect provider JWKS endpoint. | | config.oidc.opTokenEndpoint | string | `nil` | URL of your OpenID Connect provider token endpoint. | | config.oidc.opUserEndpoint | string | `nil` | URL of your OpenID Connect provider userinfo endpoint. | | config.oidc.pkceEnabled | boolean | `false` | If true, use PKCE (Proof Key of Code Exchange) to prevent auth code interception. | | config.oidc.promptLogin | boolean | `false` | If true, require the authorization server to reauthenticate the user even if the user is already authenticated. | | config.oidc.relyingPartId | string | `nil` | Relying Party ID. | | config.oidc.rpIdpSignKey | string | `nil` | The key used to sign ID tokens when using an RSA sign algorithm. | | config.oidc.rpScopes | string | `"openid allatclaims"` | The OpenID Connect scopes to request during login. | | config.oidc.rpSignAlgo | string | `"RS256"` | Signature algorithm. Allowed values: `RS256` and `HS256`. | | config.oidc.userFlagsByGroupIsActive | string | `nil` | Users will be marked as active only if they belong to one or more of the provided group(s). Use the access group delimiter to separate multiple group names. | | config.oidc.userFlagsByGroupIsSuperuser | string | `nil` | Users will be marked as superusers only if they belong to one or more of the provided group(s). Use the access groups delimiter to separate multiple group names. | | config.oidc.verifySsl | boolean | `true` | Controls whether the OpenID Connect client verifies the SSL certificate of the OP responses. | #### SDM Portal - SAML Authentication Configuration | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | config.saml | object | `-` | Configure authentication with SAML. | | config.saml.accessDenyGroup | string | `nil` | Authentication will fail for any user that belongs to one or more of the provided group(s). Use the access groups delimiter to separate multiple group names. | | config.saml.accessRequireGroup | string | `nil` | Authentication will fail for any user that does not belong to one or more of the provided group(s). Use the access groups delimiter to separate multiple group names. | | config.saml.allowUnsolicited | bool | `false` | Allow unsolicited responses from IdP. | | config.saml.enabled | boolean | `false` | Enable/disable SAML authentication. | | config.saml.entityId | string | `nil` | Entity ID. | | config.saml.mapClaimAccessGroupsDelimiter | string | `nil` | Character to split the User Access Groups on. Optional. The maximum length of the delimiter is 2 characters. These characters must not be used in any access group name. | | config.saml.mapClaimEmail | string | `"email"` | The claim containing the unique email address. | | config.saml.mapClaimFirstName | string | `"given_name"` | The claim containing the user's first name. | | config.saml.mapClaimGroups | string | `"group"` | The claim that contains the list of user's groups. | | config.saml.mapClaimGroupsDelimiter | string | `nil` | Character to split the Groups string on. Optional. | | config.saml.mapClaimLastName | string | `"family_name"` | The claim containing the user's last name. | | config.saml.mapClaimUsername | string | `"unique_name"` | The claim containing the unique username. | | config.saml.userFlagsByGroupIsActive | string | `nil` | Users will be marked as active only if they belong to one or more of the provided group(s). Use the access groups delimiter to separate multiple group names. | | config.saml.userFlagsByGroupIsSuperuser | string | `nil` | Users will be marked as superusers only if they belong to one or more of the provided group(s). Use the access groups delimiter to separate multiple group names. | #### SDM Portal Secret Configuration Values | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | secrets | object | `-` | Secret configuration values for Spectra Detect Portal | | secrets.clickhouse | string | `-` | Custom secret name for Clickhouse credentials. | | secrets.clickhouse.customSecretName | string | `nil` | Custom secret name for clickhouse credentials. If umbrella chart is used for deployment, this value is ignored, and the value from global.secrets.clickhouse.customSecretName will be used as a secret name. | | secrets.ldap | object | `-` | LDAP secret configuration values | | secrets.ldap.cacert.caCertificate | string | `nil` | LDAP CA certificate in PEM format. Creates a Secret with the `ca.pem` key. Can be set with `--set-file secrets.ldap.cacert.caCertificate=/path/to/file/ca_certificate.pem`. | | secrets.ldap.cacert.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `caCertificate` value parameters. WARNING: Use this for convenience/testing only. | | secrets.ldap.cacert.customSecretName | string | `nil` | Custom secret name. If umbrella chart is used for deployment, this value is ignored, and the value from global.secrets.ldap.cacert.customSecretName will be used as a secret name. | | secrets.ldap.cert.certificate | string | `nil` | LDAP client certificate in PEM format. Creates a Secret with the `cert.pem` key. Can be set with `--set-file secrets.ldap.cert.certificate=/path/to/file/certificate.pem`. | | secrets.ldap.cert.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `certificate` value parameters. WARNING: Use this for convenience/testing only. | | secrets.ldap.cert.customSecretName | string | `nil` | Custom secret name. If umbrella chart is used for deployment, this value is ignored, and the value from global.secrets.ldap.cert.customSecretName will be used as a secret name. | | secrets.ldap.credentials.bindDn | string | `""` | LDAP Bind DN. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.ldap.credentials.bindPassword | string | `""` | LDAP Bind Password. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.ldap.credentials.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `bindDn` and `bindPassword` parameters. WARNING: Use this for convenience/testing only. | | secrets.ldap.credentials.customSecretName | string | `nil` | Custom secret name. If umbrella chart is used for deployment, this value is ignored, and the value from global.secrets.ldap.credentials.customSecretName will be used as a secret name. | | secrets.oidc | object | `-` | Secret configuration values for setting the OIDC client credentials. | | secrets.oidc.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `rpClientId` and `rpClientSecret` parameters. WARNING: Use this for convenience/testing only. | | secrets.oidc.customSecretName | string | `nil` | Custom secret name for OIDC credentials. If umbrella chart is used for deployment, this value is ignored, and the value from global.secrets.oidc.customSecretName will be used as a secret name. | | secrets.oidc.rpClientId | string | `""` | OpenID Connect client ID provided by OpenID Connect provider. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.oidc.rpClientSecret | string | `""` | OpenID Connect client secret provided by OpenID Connect provider. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.saml | object | `-` | Secret configuration values for setting the SAML integration. | | secrets.saml.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `metadata` value. WARNING: Use this for convenience/testing only. | | secrets.saml.customSecretName | string | `nil` | Secret name containing SAML federation metadata under the `metadata.xml` key. | | secrets.saml.metadata | string | `nil` | SAML federation metadata XML. Creates a Secret with the `metadata.xml` key. Can be set with `--set-file secrets.saml.metadata=/path/to/file/metadata.xml`. | | secrets.tiCloud | object | `-` | Secret configuration values for setting the Spectra Intelligence. | | secrets.tiCloud.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `username` and `password` parameters. WARNING: Use this for convenience/testing only. | | secrets.tiCloud.customSecretName | string | `nil` | Custom secret name for Spectra Intelligence Cloud credentials. If umbrella chart is used for deployment, this value is ignored, and the value from global.secrets.tiCloud.customSecretName will be used as a secret name. | | secrets.tiCloud.password | string | `""` | Spectra Intelligence password. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.tiCloud.username | string | `""` | Spectra Intelligence username. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.tiCloudProxy | object | `-` | Secret configuration values for setting the Spectra Intelligence credentials when proxy is used. | | secrets.tiCloudProxy.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `username` and `password` parameters. WARNING: Use this for convenience/testing only. | | secrets.tiCloudProxy.customSecretName | string | `nil` | Custom secret name for Spectra Intelligence Cloud proxy credentials. If umbrella chart is used for deployment, this value is ignored, and the value from global.secrets.tiCloudProxy.customSecretName will be used as a secret name. | | secrets.tiCloudProxy.password | string | `""` | Cloud proxy password. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.tiCloudProxy.username | string | `""` | Cloud proxy username. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | #### SDM Portal Component Configuration Values | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | affinity | object | `{}` | Affinity for pod scheduling. Allows to constrain which nodes your pod can be scheduled on based on node labels or ensure pods are co-located (or isolated) from other pods. | | autoscaling | object | `-` | Autoscaling configuration values. | | autoscaling.cooldownPeriod | int | `180` | The period to wait after the last trigger reported active before scaling the resource back to 0, in seconds. | | autoscaling.enabled | bool | `false` | Enable/disable autoscaling. | | autoscaling.maxReplicas | int | `8` | Maximum number of replicas that can be deployed when scaling in enabled. | | autoscaling.minReplicas | int | `1` | Minimum number of replicas that need to be deployed. | | autoscaling.pollingInterval | int | `10` | Interval to check each trigger, in seconds. | | autoscaling.scaleDown | object | `-` | ScaleDown configuration values. | | autoscaling.scaleDown.stabilizationWindow | int | `180` | Number of continuous seconds in which the scaling condition is not met. When this is reached, scale down is started. | | autoscaling.scaleUp | object | `-` | ScaleUp configuration values. | | autoscaling.scaleUp.numberOfPods | int | `1` | Number of pods that can be scaled in the defined period. | | autoscaling.scaleUp.period | int | `30` | Interval in which the numberOfPods value is applied. | | autoscaling.scaleUp.stabilizationWindow | int | `15` | Number of continuous seconds in which the scaling condition is met. When this is reached, scale up is started. | | autoscaling.triggerCPUValue | int | `75` | CPU value (in percentage), which will cause scaling when reached. The percentage is taken from the resource.limits.cpu value. Limits have to be set up. | | clickhouse.releaseName | string | `""` | Clickhouse release name, required when deployment is not done with umbrella. | | clusterDomainName | string | `"cluster.local"` | Cluster domain name. If umbrella is used for deployment, this value is ignored, and global.clusterDomainName is applied. | | image | object | `-` | Configuration values of the SDM Portal image. | | image.pullPolicy | string | `"Always"` | Image pull policy. Options: Always, IfNotPresent, Never. | | image.tag | string | `"6.0.0-24"` | Image tag. | | imagePullSecrets | list | `[{"name":"rl-registry-key"}]` | Set of stored credentials (authentication tokens) that allows Kubernetes node to "log in" to a private container registry to pull restricted images. Each item must be an object with a 'name' key. | | ingress | object | `-` | Ingress configuration values. | | ingress.annotations | object | `{}` | Custom annotations to fine-tune the Ingress Controller behavior. | | ingress.className | string | `"nginx"` | IngressClass that will handle this resource. Must match an existing 'IngressClass' in the cluster. | | ingress.enabled | bool | `true` | Enable/disable the creation of the Ingress resource. | | ingress.host | string | `""` | The Fully Qualified Domain Name (FQDN) for the application. Required if 'enabled' is true. | | ingress.paths[n] | object | `-` | A list of paths for this host. Each path must have a path and a pathType. For the root path, use "/" with pathType "Prefix". | | ingress.paths[n].path | string | `"/"` | The URL path that this rule applies to. A single forward slash (/) represents the root or "catch-all" path. | | ingress.paths[n].pathType | string | `"Prefix"` | Determines how the Ingress controller matches the URL path. 'Prefix' matches based on a URL path prefix split by '/'. This is the most common and flexible setting for web applications. | | ingress.tls | object | `-` | TLS / SSL Configuration. | | ingress.tls.certificateArn | string | `""` | The Amazon Resource Name (ARN) of the certificate for AWS Load Balancer Controller. Only applicable when 'className' is 'alb'. | | ingress.tls.issuer | string | `""` | The name of the cert-manager ClusterIssuer or Issuer to request certificates from. | | ingress.tls.issuerKind | string | `"Issuer"` | The resource type of the issuer. Typically, 'Issuer' (namespace-scoped) or 'ClusterIssuer' (cluster-wide). | | initImage | object | `-` | Configuration values of the image used for SDM Portal initialization. | | initImage.pullPolicy | string | `"Always"` | Image pull policy. Options: Always, IfNotPresent, Never. | | initImage.tag | string | `"6.0.0-14"` | Image tag. | | loki.releaseName | string | `""` | Loki release name, required when deployment is not done with umbrella. | | nodeSelector | object | `{}` | Node labels for pod assignment. Pods will only be scheduled to nodes that match all labels defined here. | | postfix.releaseName | string | `""` | Postfix release name, required when deployment is not done with umbrella. | | postgres.customSecretName | string | `nil` | Postgres custom secret name. If umbrella chart is used for deployment, this value is ignored, and the value from global.postgresCustomSecretName will be used as a secret name. | | postgres.releaseName | string | `""` | Postgres release name, required when deployment is not done with umbrella. | | prometheus.releaseName | string | `""` | Prometheus configuration release name, required when deployment is not done with umbrella. | | rabbitmq.customAdminSecretName | string | `nil` | RabbitMQ custom admin secret name. If umbrella chart is used for deployment, this value is ignored, and the value from global.rabbitmqAdminCustomSecretName will be used as a secret name. | | rabbitmq.customSecretName | string | `nil` | RabbitMQ custom secret name. If umbrella chart is used for deployment, this value is ignored, and the value from global.rabbitmqCustomSecretName will be used as a secret name. | | rabbitmq.releaseName | string | `""` | Rabbitmq release name, required when deployment is not done with umbrella. | | replicaCount | int | `1` | Number of desired pod instances. Ignored when autoscaling is enabled. | | resources | object | `-` | Resource requests and limits for the container. | | resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | resources.limits.cpu | string | `"3000m"` | CPU limit. Throttling occurs if the container exceeds this value. | | resources.limits.memory | string | `"4Gi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | resources.requests.cpu | string | `"2000m"` | CPU request. | | resources.requests.memory | string | `"2Gi"` | Memory request. | | seaweedfs.releaseName | string | `""` | SeaweedFS release name, required when deployment is not done with umbrella. | | secrets.ldap.cacert | object | `-` | Secret containing PEM encoded file with CA certificate under the `ca.pem` key. | | secrets.ldap.cert | object | `-` | Secret containing PEM encoded certificate for client authentication PEM under the `cert.pem` key. | | secrets.ldap.credentials | object | `-` | Secret containing LDAP bind (`bindDn` and `bindPassword`) credentials. | | serviceAccount | object | `-` | Service Account configuration values. | | serviceAccount.annotations | object | `{}` | Additional annotations to add to the ServiceAccount. | | serviceAccount.create | bool | `true` | Specifies whether a ServiceAccount should be created. If false, an existing ServiceAccount name should be provided in `name` variable. | | serviceAccount.name | string | `nil` | The name of the ServiceAccount to use. If not set and create is true, default account name is used. | | useReloader | string | `nil` | Whether to enable Reloader annotations. When defined, this value takes precedence over global.useReloader. | ### Celery Worker #### Celery Worker Configuration Values | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | affinity | object | `{}` | Affinity for pod scheduling. Allows to constrain which nodes your pod can be scheduled on based on node labels or ensure pods are co-located (or isolated) from other pods. | | autoscaling | object | `-` | Autoscaling configuration values. | | autoscaling.cooldownPeriod | int | `180` | The period to wait after the last trigger reported active before scaling the resource back to 0, in seconds. | | autoscaling.enabled | bool | `false` | Enable/disable autoscaling. | | autoscaling.maxReplicas | int | `8` | Maximum number of replicas that can be deployed when scaling in enabled. | | autoscaling.minReplicas | int | `1` | Minimum number of replicas that need to be deployed. | | autoscaling.pollingInterval | int | `10` | Interval to check each trigger, in seconds. | | autoscaling.scaleDown | object | `-` | ScaleDown configuration values. | | autoscaling.scaleDown.stabilizationWindow | int | `180` | Number of continuous seconds in which the scaling condition is not met. When this is reached, scale down is started. | | autoscaling.scaleUp | object | `-` | ScaleUp configuration values. | | autoscaling.scaleUp.numberOfPods | int | `1` | Number of pods that can be scaled in the defined period. | | autoscaling.scaleUp.period | int | `30` | Interval in which the numberOfPods value is applied. | | autoscaling.scaleUp.stabilizationWindow | int | `15` | Number of continuous seconds in which the scaling condition is met. When this is reached, scale up is started. | | autoscaling.targetInputQueueSize | int | `10` | Number of messages in backlog to trigger scaling on. Must be greater than 0. | | clickhouse.releaseName | string | `""` | Clickhouse release name, required when deployment is not done with umbrella. | | image | object | `-` | Configuration values of the SDM celery worker image. | | image.pullPolicy | string | `"Always"` | Image pull policy. Options: Always, IfNotPresent, Never. | | image.tag | string | `"6.0.0-24"` | Image tag. | | imagePullSecrets | list | `[{"name":"rl-registry-key"}]` | Set of stored credentials (authentication tokens) that allows Kubernetes node to "log in" to a private container registry to pull restricted images. Each item must be an object with a 'name' key. | | nodeSelector | object | `{}` | Node labels for pod assignment. Pods will only be scheduled to nodes that match all labels defined here. | | postfix.releaseName | string | `""` | Postfix release name, required when deployment is not done with umbrella. | | postgres.customSecretName | string | `nil` | Postgres custom secret name. If umbrella chart is used for deployment, this value is ignored, and the value from global.postgresCustomSecretName will be used as a secret name. | | postgres.releaseName | string | `""` | Postgres release name, required when deployment is not done with umbrella. | | rabbitmq.customAdminSecretName | string | `nil` | RabbitMQ custom admin secret name. If umbrella chart is used for deployment, this value is ignored, and the value from global.rabbitmqAdminCustomSecretName will be used as a secret name. | | rabbitmq.customSecretName | string | `nil` | RabbitMQ custom secret name. If umbrella chart is used for deployment, this value is ignored, and the value from global.rabbitmqCustomSecretName will be used as a secret name. | | rabbitmq.releaseName | string | `""` | Rabbitmq release name, required when deployment is not done with umbrella. | | replicaCount | int | `1` | Number of desired pod instances. Ignored when autoscaling is enabled. | | resources | object | `-` | Resource requests and limits for the container. | | resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | resources.limits.cpu | string | `"3000m"` | CPU limit. Throttling occurs if the container exceeds this value. | | resources.limits.memory | string | `"4Gi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | resources.requests.cpu | string | `"1000m"` | CPU request. | | resources.requests.memory | string | `"2Gi"` | Memory request. | | seaweedfs.releaseName | string | `""` | SeaweedFS release name, required when deployment is not done with umbrella. | | secrets.clickhouse | object | `-` | Secret configuration values for setting the clickhouse credentials. | | secrets.clickhouse.customSecretName | string | `nil` | Custom secret name for clickhouse credentials. If umbrella chart is used for deployment, this value is ignored, and the value from global.secrets.clickhouse.customSecretName will be used as a secret name. | | secrets.oidc | object | `-` | Secret configuration values for setting the OIDC credentials. | | secrets.oidc.customSecretName | string | `nil` | Custom secret name for OIDC credentials. If umbrella chart is used for deployment, this value is ignored, and the value from global.secrets.oidc.customSecretName will be used as a secret name. | | secrets.tiCloud | object | `-` | Secret configuration values for setting the Spectra Intelligence cloud credentials. | | secrets.tiCloud.customSecretName | string | `nil` | Custom secret name for Spectra Intelligence credentials. If umbrella chart is used for deployment, this value is ignored, and the value from global.secrets.tiCloud.customSecretName will be used as a secret name. Otherwise, if left unset, default secret name will be used. | | secrets.tiCloudProxy | object | `-` | Secret configuration values for setting the Spectra Intelligence cloud proxy credentials. | | secrets.tiCloudProxy.customSecretName | string | `nil` | Custom secret name for Spectra Intelligence proxy credentials. If umbrella chart is used for deployment, this value is ignored, and the value from global.secrets.tiCloudProxy.customSecretName will be used as a secret name. Otherwise, if left unset, default secret name will be used. | | serviceAccount | object | `-` | Service Account configuration values. | | serviceAccount.annotations | object | `{}` | Additional annotations to add to the ServiceAccount. | | serviceAccount.create | bool | `true` | Specifies whether a ServiceAccount should be created. If false, an existing ServiceAccount name should be provided in `name` variable. | | serviceAccount.name | string | `nil` | The name of the ServiceAccount to use. If not set and create is true, default account name is used. | | useReloader | string | `nil` | Whether to enable Reloader annotations. When defined, this value takes precedence over global.useReloader. | ### Data Change Service (DCS) #### DCS Configuration Values | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | clickhouse.releaseName | string | `""` | Clickhouse release name, required when deployment is not done with umbrella. | | image | object | `-` | Configuration values of the SDM celery worker image. | | image.pullPolicy | string | `"Always"` | Image pull policy. Options: Always, IfNotPresent, Never. | | image.tag | string | `"1.31.0-1"` | Image tag. | | imagePullSecrets | list | `[{"name":"rl-registry-key"}]` | Set of stored credentials (authentication tokens) that allows Kubernetes node to "log in" to a private container registry to pull restricted images. Each item must be an object with a 'name' key. | | postfix.releaseName | string | `""` | Postfix release name, required when deployment is not done with umbrella. | | postgres.customSecretName | string | `nil` | Postgres custom secret name. If umbrella chart is used for deployment, this value is ignored, and the value from global.postgresCustomSecretName will be used as a secret name. | | postgres.releaseName | string | `""` | Postgres release name, required when deployment is not done with umbrella. | | resources | object | `-` | Resource requests and limits for the container. | | resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | resources.limits.cpu | string | `"2000m"` | CPU limit. Throttling occurs if the container exceeds this value. | | resources.limits.memory | string | `"2Gi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | resources.requests.cpu | string | `"500m"` | CPU request. | | resources.requests.memory | string | `"512Mi"` | Memory request. | | secrets.clickhouse | object | `-` | Secret configuration values for setting the clickhouse credentials. | | secrets.clickhouse.customSecretName | string | `nil` | Custom secret name for clickhouse credentials. If umbrella chart is used for deployment, this value is ignored, and the value from global.secrets.clickhouse.customSecretName will be used as a secret name. | | secrets.tiCloud | object | `-` | Secret configuration values for setting the Spectra Intelligence cloud credentials. | | secrets.tiCloud.customSecretName | string | `nil` | Custom secret name for Spectra Intelligence cloud credentials. If umbrella chart is used for deployment, this value is ignored, and the value from global.secrets.tiCloud.customSecretName will be used as a secret name. | | useReloader | string | `nil` | Whether to enable Reloader annotations. When defined, this value takes precedence over global.useReloader. | ### ClickHouse #### ClickHouse Secrets | Secret (when custom secret name is used) | Secret (default secret name) | Type | Description | Used in Components (Pods) | | :---- | :---- | :---- | :---- | :---- | | `` | `-clickhouse-secret` | Required when centralLogging is enabled | Basic authentication secret used to connect to Clickhouse. Secret is either created manually (clickhouse chart) or already exists. Required when central logging is enabled (config.centralLogging.enabled). | Portal, Celery, DCS, Clickhouse | #### ClickHouse Configuration Values | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | clusterDomainName | string | `"cluster.local"` | Cluster domain name. If umbrella is used for deployment, this value is ignored, and global.clusterDomainName is applied. | | database | string | `"default"` | Database name. | | host | string | `""` | Host for external Clickhouse. When configured, the Clickhouse deployment won't be created. | | image | object | `-` | Configuration values of the clickhouse image. | | image.pullPolicy | string | `"Always"` | Image pull policy. Options: Always, IfNotPresent, Never. | | image.repository | string | `"clickhouse/clickhouse-server"` | Image repository. | | image.tag | string | `"22.12"` | Image tag. | | persistence | object | `-` | Data storage configuration. | | persistence.accessModes | list | `["ReadWriteOnce"]` | Specifies the access modes for the volume. | | persistence.requestStorage | string | `"10Gi"` | The amount of storage to request. | | persistence.storageClassName | string | `nil` | Name of the StorageClass to use. | | port | int | `9000` | Clickhouse port. | | resources | object | `-` | Resource requests and limits for the container. | | resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | resources.limits.cpu | string | `"16"` | CPU limit. Throttling occurs if the container exceeds this value. | | resources.limits.memory | string | `"32Gi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | resources.requests.cpu | string | `"8"` | CPU request. | | resources.requests.memory | string | `"16Gi"` | Memory request. | | useReloader | string | `nil` | Whether to enable Reloader annotations. When defined, this value takes precedence over global.useReloader. | #### Clickhouse Secret Configuration Values | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | secrets | object | `-` | Secret configuration values for Clickhouse | | secrets.credentials.createUserSecret | bool | `true` | If `true`, the chart creates a Kubernetes Secret using the `username` and `password` parameters. WARNING: Use this for convenience/testing only. | | secrets.credentials.customSecretName | string | `nil` | Custom secret name for Clickhouse credentials. If umbrella chart is used for deployment, this value is ignored, and the value from global.secrets.clickhouse.customSecretName will be used as a secret name. | | secrets.credentials.password | string | `"chPass12345"` | Clickhouse password. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.credentials.username | string | `"chUser"` | Clickhouse username. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | ### Postfix #### Postfix Secrets | Secret (when custom secret name is used) | Secret (default secret name) | Type | Description | Used in Components (Pods) | | :---- | :---- | :---- | :---- | :---- | | `` | `-postfix-secret` | Required when related feature is enabled | Basic authentication secret for SMTP. Secret is either created manually (postfix chart) or already exists. | Postfix | #### Postfix Configuration Values | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | clusterDomainName | string | `"cluster.local"` | Cluster domain name. If umbrella is used for deployment, this value is ignored, and global.clusterDomainName is applied. | | image | object | `-` | Configuration values of the postfix image. | | image.pullPolicy | string | `"IfNotPresent"` | Image pull policy. Options: Always, IfNotPresent, Never. | | image.repository | string | `"juanluisbaptiste/postfix"` | Image repository. | | image.tag | string | `"1.8.0"` | Image tag. | | resources | object | `-` | Resource requests and limits for the container. | | resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | resources.limits.cpu | string | `"1000m"` | CPU limit. Throttling occurs if the container exceeds this value. | | resources.limits.memory | string | `"1Gi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | resources.requests.cpu | string | `"500m"` | CPU request. | | resources.requests.memory | string | `"512Mi"` | Memory request. | | smtp | object | `-` | SMTP configuration values. These values configure the upstream relay server used by the internal Postfix service. | | smtp.hostname | string | `"reversinglabs.com"` | The hostname used for the SMTP handshake. Emails will appear to originate from this domain. | | smtp.port | int | `25` | Port for the mail server. | | smtp.server | string | `"localhost"` | The FQDN or IP address of the mail server. | | smtp.useTls | bool | `true` | Whether to use TLS when connecting to the mail server. | | useReloader | string | `nil` | Whether to enable Reloader annotations. When defined, this value takes precedence over global.useReloader. | #### Postfix Secret Configuration Values | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | secrets | object | `-` | Secret configuration values for Postfix SMTP credentials. | | secrets.credentials.createUserSecret | bool | `false` | If `true`, the chart creates a Kubernetes Secret using the `username` and `password` parameters. WARNING: Use this for convenience/testing only. | | secrets.credentials.customSecretName | string | `nil` | Custom secret name for SMTP credentials. If umbrella chart is used for deployment, this value is ignored, and the value from global.secrets.postfix.customSecretName will be used as a secret name. | | secrets.credentials.password | string | `""` | SMTP password. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | | secrets.credentials.username | string | `""` | SMTP username. Only used if 'createUserSecret' is set to 'true'. WARNING: Use this for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. | ### SeaweedFS #### SeaweedFS Configuration Values | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | image.pullPolicy | string | `"IfNotPresent"` | Image pull policy. Options: Always, IfNotPresent, Never. | | image.repository | string | `"chrislusf/seaweedfs"` | Repository name. | | image.tag | string | `"3.62"` | Image tag. | | persistence | object | `-` | SeaweedFS data storage configuration. | | persistence.accessModes | list | `["ReadWriteOnce"]` | Specifies the access mode for the volume. | | persistence.requestStorage | string | `"20Gi"` | The amount of storage to request. | | persistence.storageClassName | string | `nil` | Name of the StorageClass to use. | | resources | object | `-` | Resource requests and limits for the container. | | resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | resources.limits.cpu | string | `"3000m"` | CPU limit. Throttling occurs if the container exceeds this value. | | resources.limits.memory | string | `"4Gi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | resources.requests.cpu | string | `"1000m"` | CPU request. | | resources.requests.memory | string | `"2Gi"` | Memory request. | | s3.port | int | `8333` | Port for the S3-compatible API service. Allows tools like AWS CLI, Minio, and various SDKs to interact with SeaweedFS as an object store. | | server.filerPort | int | `8888` | Port for the Filer server. Provides the filesystem interface and metadata management. | | server.masterPort | int | `9333` | Port for the Master server. Manages volume assignments and cluster coordination. | | server.maxVolumes | int | `30` | Maximum number of physical volumes this node is allowed to host. total_capacity = maxVolumes * volumeSizeLimitMB | | server.minFreeSpaceGiB | int | `1` | Minimum threshold of free disk space in GiB. If remaining space falls below this, the server stops accepting new writes to prevent disk exhaustion. | | server.volumePort | int | `8099` | Port for the Volume server. Handles the actual read/write operations for data chunks. | | server.volumeSizeLimitMB | int | `20000` | Maximum size in MB for each physical volume file before it is marked as read-only. | | useReloader | string | `nil` | Whether to enable Reloader annotations. When defined, this value takes precedence over global.useReloader. | ## RabbitMQ ### RabbitMQ Secrets | Secret (when custom secret name is used) | Secret (default secret name) | Type | Description | | :---- | :---- | :---- | :---- | | `` | `-rabbitmq-secret` | required | Basic authentication secret which contains username and password for RabbitMQ. If rabbitmq.createUserSecret is set to true, it will be created automatically from `username` and `password`. Otherwise, secret has to already exist.| | `` | `-rabbitmq-secret-admin` | optional | Basic authentication secret which contains username and password for RabbitMQ Admin. If rabbitmq.createManagementAdminSecret is set to true, it will be created automatically from `managementAdminUsername` and `managementAdminPassword`. If secret is missing or credentials given were invalid, credentials from the secret above (`-rabbitmq-secret`) are used. | ### RabbitMQ Values | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | affinity | object | `{}` | Affinity for pod scheduling. Allows to constrain which nodes your pod can be scheduled on based on node labels or ensure pods are co-located (or isolated) from other pods. | | applyRestrictedPolicy | bool | `false` | Apply restricted policy to RabbitMQ containers. If umbrella is used for deployment, this value is ignored, and global.applyRestrictedPolicy is used. | | clusterDomainName | string | `"cluster.local"` | Cluster domain name. If umbrella is used for deployment, this value is ignored, and global.clusterDomain name is applied. | | createManagementAdminSecret | bool | `true` | A user management admin secret will be created automatically with given admin username and admin password; otherwise, the secret must already exist. | | createUserSecret | bool | `true` | A user secret will be created automatically with the given username and password; otherwise, the secret must already exist. | | customAdminSecretName | string | `nil` | RabbitMQ custom admin secret name. If umbrella chart is used for deployment, this value is ignored, and the value from global.rabbitmqAdminCustomSecretName will be used as a secret name. | | customSecretName | string | `nil` | RabbitMQ custom secret name. If umbrella chart is used for deployment, this value is ignored, and the value from global.rabbitmqCustomSecretName will be used as a secret name. | | host | string | `""` | Host for external RabbitMQ. When configured, the Detect RabbitMQ cluster won't be created. | | image | object | `-` | Configuration values of the rabbitmq image. | | image.repository | string | `"rabbitmq"` | Image repository. | | image.tag | string | `"4.1.0-management"` | Image tag. | | managementAdminPassword | string | `""` | Management admin password. If left empty, defaults to `password` value. | | managementAdminUrl | string | `""` | Management Admin URL. If empty, defaults to `http://:15672`. | | managementAdminUsername | string | `""` | Management admin username. If left empty, defaults to `username` value. | | password | string | `"guest_11223"` | Password. | | persistence | object | `-` | RabbitMQ data storage configuration. | | persistence.requestStorage | string | `"5Gi"` | The amount of storage to request. | | persistence.storageClassName | string | `nil` | Name of the StorageClass to use. | | port | int | `5672` | RabbitMQ port. | | replicas | int | `1` | Number of desired pod instances. | | resources | object | `-` | Resource requests and limits for the container. | | resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | resources.limits.cpu | string | `"2"` | CPU limit. Throttling occurs if the container exceeds this value. | | resources.limits.memory | string | `"2Gi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | resources.requests.cpu | string | `"1"` | CPU request. | | resources.requests.memory | string | `"2Gi"` | Memory request. | | useQuorumQueues | bool | `false` | Setting this to true defines queues as quorum type (recommended for multi-replica/HA setups); otherwise, queues are classic. | | useSecureProtocol | bool | `false` | Setting this to true enables the secure AMQPS protocol for the RabbitMQ connection. | | username | string | `"guest"` | Username. | | vhost | string | `""` | Vhost that worker will use. When empty, the default `rl-detect` vhost is used. | | vhostCentralLogging | string | `""` | Vhost that central logging will use. When empty, the default `rl-detect-centrallogging` vhost is used. | | vhostSdm | string | `""` | Vhost that SDM will use. When empty, the default `rl-detect-portal` vhost is used. | ## PostgreSQL ### PostgreSQL Secrets | Secret (when custom secret name is used) | Secret (deployment with Detect chart) | Type | Description | | :---- | :---- | :---- | :---- | | `` | `-postgres-secret` | required | Basic authentication secret which contains username and password for Database. If postgres.createUserSecret is set to true, it will be created automatically from username and password. Otherwise, secret has to already exist. | ### PostgreSQL Values | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | affinity | object | `{}` | Affinity for pod scheduling. Allows to constrain which nodes your pod can be scheduled on based on node labels or ensure pods are co-located (or isolated) from other pods. | | clusterDomainName | string | `"cluster.local"` | Cluster domain name. If umbrella is used for deployment, this value is ignored, and global.clusterDomain name is applied. | | createUserSecret | bool | `true` | A user secret will be created automatically with the given username and password; otherwise, the secret must already exist. | | customSecretName | string | `nil` | Postgres custom secret name. If umbrella chart is used for deployment, this value is ignored, and the value from global.postgresCustomSecretName will be used as a secret name. | | database | string | `"tiscale"` | Database name. | | host | string | `""` | Host for external PostgreSQL. When configured, the Detect PostgreSQL cluster won't be created. | | image | object | `-` | Configuration values of the postgres image. | | image.repository | string | `"ghcr.io/cloudnative-pg/postgresql"` | Image repository. | | image.tag | string | `"17.6"` | Image tag. | | password | string | `"tiscale_11223"` | Password. | | persistence | object | `-` | Postgres data storage configuration. | | persistence.accessModes | list | `["ReadWriteOnce"]` | Specifies the access modes for the volume. | | persistence.requestStorage | string | `"5Gi"` | The amount of storage to request. | | persistence.storageClassName | string | `nil` | Name of the StorageClass to use. | | port | int | `5432` | PostgreSQL port. | | replicas | int | `1` | Number of desired pod instances. | | resources | object | `-` | Resource requests and limits for the container. | | resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | resources.limits.cpu | string | `nil` | CPU limit. Throttling occurs if the container exceeds this value. | | resources.limits.memory | string | `nil` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | resources.requests.cpu | string | `"500m"` | CPU request. | | resources.requests.memory | string | `"1Gi"` | Memory request. | | schema | string | `""` | Schema name which will be used for SDM. When empty, default name "portal" will be used. | | username | string | `"tiscale"` | Username. Required if `host` is not set, because the Detect PostgreSQL cluster will be created and this user will be set as the database owner. | ## Logging | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | alloy.enabled | bool | `true` | If enabled, the Alloy subchart will be deployed for log collection. | | grafana.enabled | bool | `true` | If enabled, the Grafana subchart will be deployed for data visualization. | | loki.enabled | bool | `true` | If enabled, the Loki subchart will be deployed. | ### Alloy | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | clusterDomainName | string | `"cluster.local"` | Cluster domain name. If umbrella is used for deployment, this value is ignored, and global.clusterDomainName is applied. | | config | object | `-` | Alloy configuration values. | | config.extraLabels | object | `{}` | Extra labels to add to all collected logs. | | config.lokiUrl | string | `""` | Custom Loki URL for log forwarding. Leave empty to use the deployed Loki instance. | | config.namespaces | list | `["default"]` | List of namespaces to collect logs from. Empty array means all namespaces. | | config.scrapeInterval | string | `"15s"` | Scrape interval for log collection. | | enabled | bool | `false` | Enable or disable the Alloy deployment. | | image | object | `-` | Configuration values of the alloy image. | | image.pullPolicy | string | `"IfNotPresent"` | Image pull policy. Options: Always, IfNotPresent, Never. | | image.repository | string | `"grafana/alloy"` | Image repository. | | image.tag | string | `"v1.11.3"` | Image tag. | | rbac.create | bool | `true` | Create RBAC resources (ClusterRole and ClusterRoleBinding for log collection). | | resources | object | `-` | Resource requests and limits for the container. | | resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | resources.limits.cpu | string | `"500m"` | CPU limit. Throttling occurs if the container exceeds this value. | | resources.limits.memory | string | `"512Mi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | resources.requests.cpu | string | `"100m"` | CPU request. | | resources.requests.memory | string | `"128Mi"` | Memory request. | | serviceAccount.annotations | object | `{}` | Annotations to add to the service account. | | serviceAccount.create | bool | `true` | Create a service account. | | serviceAccount.name | string | `""` | Name of the service account to use. If empty, a name is generated using the fullname template. | ### Grafana | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | admin.password | string | `""` | Grafana admin password. | | admin.username | string | `""` | Grafana admin username. | | clusterDomainName | string | `"cluster.local"` | Cluster domain name. If umbrella is used for deployment, this value is ignored, and global.clusterDomainName is applied. | | config | object | `-` | Configuration values. | | config.lokiUrl | string | `""` | Custom Loki URL for datasource. Leave empty to use the deployed Loki instance. | | config.workerReleaseName | string | `nil` | Spectra Detect Worker release name to represent the logs from the Worker components in the created dashboard. It is required only if logging chart is deployed without umbrella chart. In case the worker in question was deployed with umbrella chart, the release name needs to be `-wrk`, otherwise it should be ``. | | enabled | bool | `false` | Enable or disable the Grafana deployment. | | image | object | `-` | Configuration values of the grafana image. | | image.pullPolicy | string | `"IfNotPresent"` | Image pull policy. Options: Always, IfNotPresent, Never. | | image.repository | string | `"grafana/grafana"` | Image repository. | | image.tag | string | `"12.2.1"` | Image tag. | | ingress | object | `-` | Ingress configuration values. | | ingress.annotations | object | `{}` | Custom annotations to fine-tune the Ingress Controller behavior. | | ingress.className | string | `"nginx"` | IngressClass that will handle this resource. Must match an existing 'IngressClass' in the cluster. | | ingress.enabled | bool | `true` | Enable/disable the creation of the Ingress resource. | | ingress.host | string | `""` | The Fully Qualified Domain Name (FQDN) for the application. Required if 'enabled' is true. | | ingress.tls | object | `-` | TLS / SSL Configuration. | | ingress.tls.certificateArn | string | `""` | The Amazon Resource Name (ARN) of the certificate for AWS Load Balancer Controller. Only applicable when 'className' is 'alb'. | | ingress.tls.issuer | string | `""` | The name of the cert-manager ClusterIssuer or Issuer to request certificates from. | | ingress.tls.issuerKind | string | `"Issuer"` | The resource type of the issuer. Typically, 'Issuer' (namespace-scoped) or 'ClusterIssuer' (cluster-wide). | | persistence | object | `-` | Grafana data storage configuration. | | persistence.accessMode | string | `"ReadWriteOnce"` | Specifies the access mode for the volume. | | persistence.enabled | bool | `true` | Enable/disable persistent storage for Grafana data. | | persistence.size | string | `"1Gi"` | Persistent data storage size. | | persistence.storageClass | string | `""` | Name of the StorageClass to use. | | resources | object | `-` | Resource requests and limits for the container. | | resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | resources.limits.cpu | string | `"500m"` | CPU limit. Throttling occurs if the container exceeds this value. | | resources.limits.memory | string | `"512Mi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | resources.requests.cpu | string | `"100m"` | CPU request. | | resources.requests.memory | string | `"128Mi"` | Memory request. | | service | object | `-` | Service configuration for network access. | | service.port | int | `3000` | HTTP port number the service will listen on. | | service.type | string | `"ClusterIP"` | Service type determines how the service is exposed. | ### Loki | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | clusterDomainName | string | `"cluster.local"` | Cluster domain name. If umbrella is used for deployment, this value is ignored, and global.clusterDomainName is applied. | | config | object | `-` | Loki configuration values. | | config.limits.maxQuerySeries | int | `500` | Maximum number of series returned by a query. | | config.limits.maxStreamsPerUser | int | `10000` | Maximum number of streams per user. | | config.retentionPeriod | string | `"168h"` | Log retention period (e.g., "168h" for 7 days). | | config.storage.type | string | `"filesystem"` | Storage backend type (filesystem, s3, gcs). | | enabled | bool | `false` | Enable or disable the Loki deployment. | | host | string | `""` | External Loki host URL. When set, local Loki deployment is skipped and external Loki is used. | | image | object | `-` | Configuration values of the loki image. | | image.pullPolicy | string | `"IfNotPresent"` | Image pull policy. Options: Always, IfNotPresent, Never. | | image.repository | string | `"grafana/loki"` | Image repository. | | image.tag | string | `"3.5.8"` | Image tag. | | persistence | object | `-` | Loki data storage configuration. | | persistence.accessMode | string | `"ReadWriteOnce"` | Specifies the access mode for the volume. | | persistence.enabled | bool | `true` | Enable/disable persistent storage for Loki data. | | persistence.size | string | `"30Gi"` | Persistent data storage size. | | persistence.storageClass | string | `""` | Name of the StorageClass to use. | | resources | object | `-` | Resource requests and limits for the container. | | resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | resources.limits.cpu | string | `"1000m"` | CPU limit. Throttling occurs if the container exceeds this value. | | resources.limits.memory | string | `"1Gi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | resources.requests.cpu | string | `"100m"` | CPU request. | | resources.requests.memory | string | `"256Mi"` | Memory request. | | service | object | `-` | Service configuration for network access. | | service.grpcPort | int | `9096` | gRPC service port. | | service.httpPort | int | `3100` | HTTP port number the service will listen on. | | service.type | string | `"ClusterIP"` | Service type determines how the service is exposed. | ## Prometheus | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | clusterDomainName | string | `"cluster.local"` | Cluster domain name. If umbrella is used for deployment, this value is ignored, and global.clusterDomainName is applied. | | enabled | bool | `true` | Enable or disable the Prometheus configuration deployment. It provides only configuration and secrets for connecting to Prometheus. Prometheus itself is not deployed when this value is 'true'. | | namespace | string | `"monitoring"` | Namespace where the external Prometheus is deployed. | | port | int | `9090` | Prometheus port. | | releaseName | string | `"kube-prometheus-stack"` | External Prometheus release name. | ## Reloader | Key | Type | Default | Description | | :---- | :---- | :---- | :---- | | reloader.enabled | bool | `true` | If enabled, reloader will be deployed. Reloader ensures that latest configuration is applied at all time. | | reloader.reloader | object | `-` | Reloader configuration values. | | reloader.reloader.logFormat | string | `"json"` | Set the type of log format. Use "json" for structured logging or leave empty "" for plain text. | | reloader.reloader.logLevel | string | `"info"` | Set the verbosity of the logs. Options: trace, debug, info, warning, error, fatal, panic. | | reloader.reloader.resources | object | `-` | Resource requests and limits for the container. | | reloader.reloader.resources.limits | object | `-` | The maximum amount of resources the container is allowed to consume. | | reloader.reloader.resources.limits.cpu | string | `"100m"` | CPU limit. Throttling occurs if the container exceeds this value. | | reloader.reloader.resources.limits.memory | string | `"512Mi"` | Memory limit. If exceeded, the container may be terminated with an OOMKilled error. | | reloader.reloader.resources.requests | object | `-` | The minimum amount of resources the container is guaranteed. | | reloader.reloader.resources.requests.cpu | string | `"10m"` | CPU request. | | reloader.reloader.resources.requests.memory | string | `"128Mi"` | Memory limit. | --- ## Spectra Detect AWS EKS Microservices Deployment — Helm, KEDA, and RabbitMQ ## Introduction This document describes how the Spectra Detect platform is deployed and operated on Kubernetes, providing a high-volume, high-speed file processing and analysis that seamlessly integrates into existing infrastructure and effectively scales with business needs. The platform is packaged as container images and deployment is managed with Helm charts, which package Kubernetes manifests and configuration into versioned releases for consistent installs, upgrades, and rollbacks. Configuration is externalized via [ConfigMaps](./config-reference.md) and [Secrets](./config-reference.md) so behavior can be adjusted without rebuilding images, and sensitive data is stored separately with controlled access. Horizontal Pod Autoscaling may adjust replica counts based on metrics such as CPU utilization or queue size. ## Requirements - EKS version 1.34, Amazon Linux 2023 with cgroupsv2 (tested) - `Persistent Volume` provisioner supporting `ReadWriteMany` (e.g. Amazon EFS CSI). - `Persistent Volume` provisioner supporting `ReadWriteOnce` (e.g., "Amazon EBS, gp2, gp3") - Ingress controller (nginx or AWS ALB) - Helm 3 or above ## Operators and Tools ### Keda (autoscaling - optional) For Spectra Detect to autoscale Workers, [Keda](https://keda.sh/) needs to be installed on the cluster. Keda can be deployed following the official [Deploying Keda documentation](https://keda.sh/docs/2.17/deploy/). It is not required to have Keda installed to run Spectra Detect on K8s, but it is required to utilize Worker autoscaling features. ### Prometheus Operator All Worker and Integration pods have metrics exposed in Prometheus format. There is a Prometheus chart which can be deployed with umbrella. This chart is strictly configuration chart, it creates necessary resources to allow other components to connect with Prometheus. It does not deploy the Prometheus server itself, for that a separate Prometheus instance needs to be deployed. If Prometheus instance is not already present in the cluster, steps from the [Deploying Prometheus](#deploying-prometheus) section can be used to deploy it. #### Prometheus Configuration To use Prometheus with Spectra Detect, you need to configure the following values: ```yaml # Configures Prometheus monitoring for Worker pods worker: monitoring: # -- Enable/disable monitoring with Prometheus enabled: true # -- Use actual release name prometheusReleaseName: "${PROMETHEUS_RELEASE_NAME}" # Configures Prometheus monitoring for Integration pods connectorS3: monitoring: # -- Enable/disable monitoring with Prometheus. enabled: false # -- Prometheus release name. prometheusReleaseName: "${PROMETHEUS_RELEASE_NAME}" # Create necessary resources for Prometheus integration in other components (SDM) prometheus: enabled: true releaseName: "${PROMETHEUS_RELEASE_NAME}" namespace: "${PROMETHEUS_NAMESPACE}" ``` Additional information about configuration values for Prometheus chart can be found [Prometheus Configuration Reference](./config-reference.md#prometheus). #### Deploying Prometheus To deploy Prometheus, you can use the official Prometheus Helm chart. The chart can be found [here](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack). 1) Add the Prometheus community repository: ```bash helm repo add prometheus-community https://prometheus-community.github.io/helm-charts helm repo update ``` 2) Run the following command to deploy the chart in your cluster: ```sh # Set release name and namespace HELM_RELEASE_NAME="kube-prometheus-stack" NAMESPACE="monitoring" # Install the kube-prometheus-stack chart helm install "$HELM_RELEASE_NAME" --create-namespace --namespace "$NAMESPACE" -f prometheus-custom-values.yaml \ prometheus-community/kube-prometheus-stack ``` The file `prometheus-custom-values.yaml` contains custom configuration for the Prometheus chart. Example of the content of `prometheus-custom-values.yaml`: ```yaml prometheus: prometheusSpec: # Monitor all namespaces serviceMonitorNamespaceSelector: { } podMonitorNamespaceSelector: { } podMetadata: labels: app.kubernetes.io/name: prometheus app.kubernetes.io/component: prometheus app.kubernetes.io/part-of: detect # Disable Grafana since we deploy it separately grafana: enabled: false # Allow kube-state-metrics to expose these labels in kube_pod_labels kube-state-metrics: extraArgs: - >- --metric-labels-allowlist=pods=[ app.kubernetes.io/component, app.kubernetes.io/name, app.kubernetes.io/part-of, app.kubernetes.io/version, reversinglabs.com/image-tag, app.kubernetes.io/instance, helm.sh/chart ] ``` ### RabbitMQ Broker Spectra Detect Helm charts support using external RabbitMQ Brokers (like AmazonMQ), as well as deploying and using RabbitMQ cluster resources as part of a Detect deployment installed in the same namespace. Choose which option to use based on the business requirements. - **External RabbitMQ Broker** (deployed and managed outside of Spectra Detect Helmcharts) External/existing RabbitMQ Broker needs to be set up as per the broker installation guides. As an example, please check the [Amazon MQ instructions](https://docs.aws.amazon.com/amazon-mq/latest/developer-guide/getting-started-rabbitmq.html). - **RabbitMQ Operator** (deployed and managed by Spectra Detect Helm charts) Cloud native brokers can be deployed and managed by Spectra Detect Helm charts. RabbitMQ Operator needs to be installed in the K8s cluster. ```bash kubectl apply --wait -f \ https://github.com/rabbitmq/cluster-operator/releases/download/v2.6.0/cluster-operator.yml ``` | Secret (when custom secret name is used) | Secret (default secret name) | Type |Description | | :---- | :---- | :---- | :---- | | `` | `-rabbitmq-secret` | required | Basic authentication secret which contains the RabbitMQ username and password. Secret is either created manually (rabbitmq chart) or already exists. | | `` | `-rabbitmq-secret-admin` | optional | Basic authentication secret which contains the RabbitMQ Admin username and password. Secret is either created manually (rabbitmq chart) or already exists. If missing, credentials from `-rabbitmq-secret`. | For Detect application to work properly, the following vhosts are needed and will be created on deploy if missing: - Vhost for Worker Processing - Vhost for SDM - Vhost for Central Logging All vhost names can be configured; otherwise, the default names will be used. View the [RabbitMQ Configuration Reference](config-reference.md#rabbitmq) for more information about the configuration options. ### PostgreSQL Server Spectra Detect Helm charts support using external PostgreSQL Clusters (like Amazon RDS), as well as deploying and using PostgreSQL cluster resources as part of a Detect deployment. - **External PostgreSQL Server** (deployed and managed outside of Spectra Detect Helm charts) External/existing PostgreSQL server needs to be set up as per the PostgreSQL server guide. As an example, please check the Amazon RDS instructions - **CloudNativePG Operator** (deployed and managed by Spectra Detect Helm charts) CloudNativePG Operator needs to be installed in the K8s cluster. ```bash # PostgreSQL Operator - CloudNativePG (CNPG) kubectl apply --wait -f \ https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/release-1.21/releases/cnpg-1.21.1.yaml ``` | Secret (when custom secret name is used) | Secret (deployment with Detect chart) | Type | Description | | :---- | :---- | :---- | :---- | | `` | `-postgres-secret` | required | Basic authentication secret which contains the database username and password. Secret is either created manually (postgres chart) or already exists. | For Detect application to work properly, the database and schema are needed and will be created on deploy if missing. Schema is used for SDM component. Database and schema names can be configured; otherwise default names will be used. View the [PostgreSQL Configuration Reference](config-reference.md#postgresql) for more information about the configuration options. ### Reloader Secret and ConfigMap changes in SDM subcharts are handled by Reloader (https://github.com/stakater/Reloader). This is a Kubernetes controller that watches changes in ConfigMap and Secrets and does rolling upgrades on Pods with their associated Deployment, StatefulSet, DaemonSet and DeploymentConfigs. **Warning: Reloader is currently only applicable for SDM components. Worker components have internal mechanisms for applying configuration changes.** #### Deploy Reloader Reloader can be deployed with the umbrella chart if needed. 1) Add the Reloader Helm chart repository: ```sh helm repo add stakater https://stakater.github.io/stakater-charts helm repo update ``` 2) Deploy the Reloader with the umbrella chart by setting `reloader.enabled` to `true`. Currently, it is deployed by default. **Info: View the [Reloader Configuration Reference](config-reference.md#reloader) for more information about the configuration options of the Reloader deployed by the umbrella chart.** #### Reloader Usage To actually use the Reloader to automatically apply changes to Secrets and ConfigMaps, the `global.useReloader` flag must be set to `true`. If `global.useReloader` is set to false, changes to Secrets and ConfigMaps from SDM components will not be automatically applied. Additionally, Reloader can be configured for specific SDM components with `useReloader` value, which will override the `global.useReloader` setting. **Warning: If Reloader is disabled, a manual rollout restart must be performed:** ``` kubectl rollout restart deployment kubectl rollout restart statefulset ``` ## Storage ### Persistent volume There are multiple persistent volumes used by the Detect components: 1) RabbitMQ (if deployed) If RabbitMQ is deployed with the umbrella chart, it will use a persistent volume to store message data. The requested access mode is `ReadWriteOnce`, and any type of storage that supports that can be used (e.g. in AWS encrypted-gp2). Example of configuration: ```yaml rabbitmq: persistence: storageClassName: "gp2" requestStorage: "5Gi" ``` 2) Postgres (if deployed) If Postgres is deployed with the umbrella chart, it will use a persistent volume to store data. The requested access mode is `ReadWriteOnce`, and any type of storage that supports that can be used (e.g. in AWS encrypted-gp2). Example of configuration: ```yaml postgres: persistence: storageClassName: "gp2" requestStorage: "5Gi" ``` 3) Worker The `/scratch` folder is implemented as a persistent volume. Multiple services have access to the volume: - Cleanup - Health Check - Preprocessor - Processor (Standard, Retry and Preprocessor Unpacker) - Postprocessor - Receiver - TcLibs - Connector S3 (if deployed with **sharedStorage** enabled) Since multiple services (pods) are accessing the volume, the access mode of that volume has to be `ReadWriteMany`. In AWS, it is recommended to use EFS storage since it supports the requested access for pods. (More information can be found [here](#amazon-efs-remote-storage)) Example of configuration: ```yaml worker: persistence: storageClassName: "efs-sc" requestStorage: "100Gi" ``` 4) Connector S3 (if deployed) If Connector S3 is deployed, it will use a persistent volume for BoltDB. The requested access mode is `ReadWriteOnce`, and any type of storage that supports that can be used (e.g. in AWS encrypted-gp2). This volume can be configured in the following way: ```yaml connectorS3: persistence: storageClassName: "gp2" requestStorage: "5Gi" ``` If deployed with **sharedStorage** enabled, the Connector S3 will use a persistent volume created in Worker to store data (`/scratch` folder). 5) SeaweedFS (if deployed) If SeaweedFS is deployed with the umbrella chart, it will use a persistent volume for data storage. The requested access mode is `ReadWriteOnce`, and any type of storage that supports that can be used (e.g. in AWS encrypted-gp2). Example of configuration: ```yaml seaweedfs: persistence: storageClassName: "gp2" requestStorage: "50Gi" ``` 6) ClickHouse (if deployed) If ClickHouse is deployed with the umbrella chart, it will use a persistent volume for data storage. The requested access mode is `ReadWriteOnce`, and any type of storage that supports that can be used (e.g. in AWS encrypted-gp2). Example of configuration: ```yaml clickhouse: persistence: storageClassName: "gp2" requestStorage: "20Gi" ``` 7) Loki (if deployed) If Loki is deployed with the umbrella chart, it will use a persistent volume for data storage. If persistent storage is not enabled it will use `emptyDir`. The requested access mode is `ReadWriteOnce`, and any type of storage that supports that can be used (e.g. in AWS encrypted-gp2). Example of configuration: ```yaml logging: loki: persistence: enabled: true storageClass: "gp2" size: "30Gi" ``` 8) Grafana (if deployed) If Grafana is deployed with the umbrella chart, it will use a persistent volume for data storage. If persistent storage is not enabled it will use `emptyDir`. The requested access mode is `ReadWriteOnce`, and any type of storage that supports that can be used (e.g. in AWS encrypted-gp2). Example of configuration: ```yaml logging: grafana: persistence: enabled: true storageClass: "gp2" size: "1Gi" ``` ### Ephemeral volume There are multiple ephemeral volumes used by the Detect components: 1) Connector S3 (if deployed) If Connector S3 is deployed and **sharedStorage** is not used, it will use an ephemeral volume for temporary storage of files downloaded from S3. The requested access mode is `ReadWriteOnce`, and any type of storage that supports that can be used (e.g. in AWS encrypted-gp2). The storage class name is empty by default (value `tmp.storageClassName`). If not overridden, `emptyDir` will be used for storage. Example of configuration: ```yaml connectorS3: tmp: storageClassName: "gp2" requestStorage: "10Gi" ``` 2) Worker The `/tc-scratch` folder is used as an ephemeral volume for temporary processing files in the following processing components: - Processor - Retry Processor - Preprocessor Unpacker Each component has its own ephemeral volume, but the configuration is shared. The requested access mode is `ReadWriteOnce`, and any type of storage that supports that can be used (e.g. in AWS encrypted-gp2). The storage class name is empty by default (value `tcScratch.storageClassName`). If not overridden, `emptyDir` will be used for storage. Example of configuration: ```yaml worker: tcScratch: storageClassName: "gp2" requestStorage: "100Gi" ``` ### Amazon EFS Remote Storage If you are running Kubernetes on Amazon EKS, you can use Amazon EFS storage for the shared storage. You will need to: 1. Install Amazon [EFS CSI Driver](https://docs.aws.amazon.com/eks/latest/userguide/efs-csi.html) on the cluster to use EFS 2. Create EFS file system via Amazon EFS console or [command line](https://github.com/kubernetes-sigs/aws-efs-csi-driver/blob/master/docs/efs-create-filesystem.md) 3. Set Throughput mode to Elastic or Provisioned for higher throughput levels 4. Add [mount targets](https://docs.aws.amazon.com/efs/latest/ug/accessing-fs.html) for the node's subnets 5. Create a [storage class](https://github.com/kubernetes-sigs/aws-efs-csi-driver/blob/master/examples/kubernetes/dynamic_provisioning/README.md#dynamic-provisioning) for Amazon EFS ## Getting Started ReversingLabs Spectra Detect Helm charts and container images are available at **registry.reversinglabs.com**. In order to connect to the registry, you need to use the ReversingLabs Spectra Intelligence account. ```bash helm registry login registry.reversinglabs.com -u "${RL_SPECTRA_INTELLIGENCE_USERNAME}" ``` If you want to see which versions of the charts are available in the registry, you can use a tool like [Skopeo](https://github.com/containers/skopeo) to log in to the registry and list the versions: ```bash skopeo login registry.reversinglabs.com -u "${RL_SPECTRA_INTELLIGENCE_USERNAME}" skopeo list-tags docker://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform eg. { "Repository": "registry.reversinglabs.com/detect/charts/detect-suite/detect-platform", "Tags": [ "5.7.0-0.beta.3" ] } ``` ### List of Detect Worker Components | Component | Image | Mandatory/Optional | Scaling * (see [Appendix](#appendix)) | |-----------------------------------|------------------------------|--------------------|---------------------------------------| | `-wrk-auth` | `rl-detect-auth` | Optional | N/A | | `-wrk-auth-proxy` | `nginx` | Optional | N/A | | `-wrk-receiver` | `rl-detect-receiver` | Mandatory | CPU | | `-wrk-preproc` | `rl-detect-preprocessor` | Optional | QUEUE | | `-wrk-proc` | `rl-processor` | Mandatory | QUEUE | | `-wrk-proc-retry` | `rl-processor` | Mandatory | QUEUE | | `-wrk-preproc-unp` | `rl-processor` | Optional | QUEUE | | `-wrk-tclibs` | `rl-tclibs` | Mandatory | CPU | | `-wrk-postproc` | `rl-detect-postprocessor` | Mandatory | QUEUE | | `-wrk-cloud-cache` | `rl-cloud-cache` | Optional | N/A | | `-wrk-cleanup-job` | `rl-detect-utilities` | Mandatory | N/A | | `-wrk-health-job` | `rl-detect-utilities` | Mandatory | N/A | | `-wrk-config-mng` | `rl-detect-config-manager` | Optional | N/A | ### List of Detect Integration Components | Component | Image | Mandatory/Optional | Scaling * (see [Appendix](#appendix)) | |-------------------------------|---------------------|--------------------|---------------------------------------| | `-connector-s3` | `rl-integration-s3` | Optional | N/A | ### List of Detect Manager Components | Component | Image | Mandatory/Optional | Scaling * (see [Appendix](#appendix)) | |------------------------------------|------------------------------|--------------------|---------------------------------------| | `-sdm-portal` | `rl-detect-portal` | Optional | CPU | | `-celery-worker` | `rl-detect-portal-celery` | Optional | QUEUE | | `-celery-scheduler` | `rl-detect-portal-celery` | Optional | N/A | | `-sdm-data-change` | `rl-detect-data-change` | Optional | N/A | | `-clickhouse` | `clickhouse/clickhouse-server` | Optional | N/A | | `-postfix` | `juanluisbaptiste/postfix` | Optional | N/A | | `-seaweedfs` | `chrislusf/seaweedfs` | Optional | N/A | ### Deploying Detect using Helm Charts 1. Create a namespace or use an existing one ```bash kubectl create namespace detect # Namespace name is arbitrary ``` 2. Set up the Registry Secret to allow Kubernetes to pull container images. The Kubernetes secret `rl-registry-key` containing the user's Spectra Intelligence credentials needs to be created in the namespace where Detect will be installed. 2.1. The secret can either be created via Detect Helm chart - `registry.createRegistrySecret`: needs to be set to true (default) - `registry.authSecretName`: value needs to be the Spectra Intelligence account username. - `registry.authSecretPassword`: value needs to be the Spectra Intelligence account password. 2.2. Can be managed outside the Helm release - `registry.createRegistrySecret`: value should be set to false. You can create the secret manually by using the following command ```bash kubectl apply -n "detect" -f - <<EOF apiVersion: v1 kind: Secret metadata: name: "rl-registry-key" type: kubernetes.io/dockerconfigjson data: .dockerconfigjson: $(echo -n '{"auths": {"registry.reversinglabs.com": {"auth": "'$(echo -n "${SPECTRA_INTELLIGENCE_USERNAME}:${SPECTRA_INTELLIGENCE_PASSWORD}" | base64)'"}}}' | base64 | tr -d '\n') EOF ``` 3. Install a Spectra Detect Chart Run the following command using a deployment name of your choice (max 30 characters). For more details regarding values, please reference the [Appendix](#appendix). ```bash helm install "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform \ --version "${DETECT_HELM_CHART_VERSION}" --namespace "${NAMESPACE}" -f values.yaml ``` 4. Configure Ingress Controller In order to be able to access Spectra Detect endpoints from outside the cluster, and in order for Worker pod to be able to connect to the Spectra Detect Manager, an Ingress Controller (like AWS ALB or Nginx Controller) must be configured on the K8s cluster. Follow the official installation guides for the controllers: - [AWS Load Balancer Controller](https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.13/deploy/installation/) - [Nginx Controller](https://docs.nginx.com/nginx-ingress-controller/installation/installing-nic/) This example shows how to configure Ingress for Worker pod using [AWS ALB Controller](https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/) and how to use [External DNS](https://kubernetes-sigs.github.io/external-dns/v0.15.0/) to automatically create DNS records in AWS Route53. > Ingress values configuration example using AWS ALB Controller ```yaml worker: ingress: annotations: alb.ingress.kubernetes.io/backend-protocol: HTTP alb.ingress.kubernetes.io/certificate-arn: <> alb.ingress.kubernetes.io/group.name: detect alb.ingress.kubernetes.io/healthcheck-interval-seconds: "15" alb.ingress.kubernetes.io/healthcheck-path: /livez alb.ingress.kubernetes.io/healthcheck-port: "80" alb.ingress.kubernetes.io/healthcheck-protocol: HTTP alb.ingress.kubernetes.io/healthcheck-timeout-seconds: "5" alb.ingress.kubernetes.io/healthy-threshold-count: "2" alb.ingress.kubernetes.io/load-balancer-attributes: idle_timeout.timeout_seconds=600 alb.ingress.kubernetes.io/manage-backend-security-group-rules: "true" alb.ingress.kubernetes.io/scheme: internal alb.ingress.kubernetes.io/security-groups: <> alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-FS-1-2-Res-2020-10 alb.ingress.kubernetes.io/success-codes: 200,301,302,404 alb.ingress.kubernetes.io/target-group-attributes: stickiness.enabled=true,stickiness.lb_cookie.duration_seconds=1200 alb.ingress.kubernetes.io/target-type: ip alb.ingress.kubernetes.io/unhealthy-threshold-count: "2" external-dns.alpha.kubernetes.io/hostname: detect-platform-worker.example.com external-dns.alpha.kubernetes.io/ingress-hostname-source: annotation-only className: alb enabled: true host: detect-platform-worker.example.com ``` Additionally, two more Ingresses are used if Logging and SDM are deployed: - Ingress for Grafana - Ingress for SDM They can be configured similarly to the worker ingress. ### Updating Detect using Helm Charts Before running the upgrade, you can use the helm `diff upgrade` command to see the changes that will occur in the Kubernetes manifest files. [Helm Diff Plugin](https://github.com/helmfile/helmfile) must be installed to utilize the `diff` feature. You can install Helm Diff using the following command: ```bash # Install plugin helm plugin install https://github.com/databus23/helm-diff # Run diff command helm diff upgrade "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform \ --version "${DETECT_CHART_VERSION}" \ --namespace "${NAMESPACE}" \ -f values.yaml # Check the Helm chart readme beforehand if you want helm show readme oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform --version "${DETECT_CHART_VERSION}" # Run upgrade helm upgrade "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform \ --version "${DETECT_CHART_VERSION}" \ --namespace "${NAMESPACE}" \ -f values.yaml ``` ### Uninstalling Detect ```bash helm uninstall detect -n "${NAMESPACE}" ``` ## Spectra Detect Worker ### Introduction Spectra Detect Worker analyzes files submitted via the Worker API (or via connector) and produces a detailed analysis report for every file using the built-in Spectra Core static analysis engine. Worker is composed of the following components: | Component | Description | | :---- | :---- | | `wrk-auth` | Authentication service | | `wrk-auth-proxy` | Authentication reverse proxy | | `wrk-receiver` | File receiver service | | `wrk-preproc` | File preprocessing service | | `wrk-proc` | File processing service | | `wrk-proc-retry` | Retry and large file processing service | | `wrk-preproc-unp` | Preprocessor unpacking service | | `wrk-tclibs` | Tclibs service | | `wrk-postproc` | Postprocessing service | | `wrk-cloud-cache` | Cloud cache service | | `wrk-cleanup-job` | Cleanup job for old files and tasks | | `wrk-health-job` | Health check job | | `wrk-config-mng` | Configuration management service (Worker configuration given to SDM) | ### Component Deployment Conditions Prerequisite for all components: `worker.enabled: true` must be set in the values file. | Component | Deployment Condition | Description | | :---- | :---- | :---- | | `wrk-auth` | `worker.configuration.authentication.enabled: true` and `worker.configuration.authentication.externalAuthUrl: ""` | Deployed only when authentication is enabled and external authentication service is not used. | | `wrk-auth-proxy` | `worker.configuration.authentication.enabled: true` and `worker.ingress.className != "nginx"` | Deployed only if authentication is enabled and ingress doesn't support sub-request auth (currently if not nginx ingress). | | `wrk-receiver` | `-` | Always deployed. | | `wrk-preproc` | `worker.configuration.cloud.enabled: true` and (`worker.configuration.cloudAutomation.spexUpload.enabled: true` or `worker.configuration.cloudAutomation.dataChangeSubscribe: true`) | Deployed only when Cloud and either AV file analysis or data change subscribe is enabled. | | `wrk-proc` | `-` | Always deployed. | | `wrk-proc-retry` | `-` | Always deployed. | | `wrk-preproc-unp` | `worker.configuration.cloud.enabled: true` and`worker.configuration.cloudAutomation.spexUpload.enabled: true` and `worker.configuration.cloudAutomation.spexUpload.scanUnpackedFiles` | Deployed only when Cloud is enabled, AV file analysis is enabled and scanning of unpacked files is enabled. | | `wrk-tclibs` | `-` | Always deployed. | | `wrk-postproc` | `-` | Always deployed. | | `wrk-cloud-cache` | `worker.configuration.cloud.enabled: true` and`worker.configuration.cloudCache.enabled: true`| Deployed only if cloud and cloud cache are enabled. | | `wrk-cleanup-job` | `-` | Always deployed. | | `wrk-health-job` | `-` | Always deployed. | | `wrk-config-mng` | `worker.configManager.enabled: true` and (SDM deployed with umbrella or `worker.sdmPortal.urlOverride` configured)| Deployed if config manager feature is enabled and if SDM is properly configured (either SDM deployed with umbrella along with the worker or SDM Portal URL override is provided). | ### Dependencies RabbitMQ and PostgreSQL are needed for Worker to function properly. These components can be externally provided or deployed as part of the same Detect Platform chart. View the [RabbitMQ](#rabbitmq-broker) and [PostgreSQL](#postgresql-server) documentation for more information. ### Application Secrets Multiple secrets are needed for proper Worker functionality. The list of all the secrets can be found in the [Worker Secrets](./config-reference.md#worker-secrets) section. All Worker secret names can be customized and they can be created with Helm Chart. More information about secret customization can be found [here](#secrets). ### Configuration Reference Configuration for Worker components can be found in the [Configuration Reference](config-reference.md#processing). #### Default Resource Configuration | Component | CPU Request | CPU Limit | Memory Request | Memory Limit | | :---- | :---- | :---- | :---- | :---- | | wrk-auth | `500m` | `4000m` | `128Mi` | `256Mi` | | wrk-auth-proxy | `250m` | `2000m` | `128Mi` | `512Mi` | | wrk-receiver | `2000m` | `5000m` | `1Gi` | `8Gi` | | wrk-preproc | `1000m` | `4000m` | `1Gi` | `4Gi` | | wrk-proc | `4000m` | `~` | `4Gi` | `32Gi` | | wrk-proc-retry | `4000m` | `~` | `8Gi` | `64Gi` | | wrk-preproc-unp | `4000m` | `~` | `4Gi` | `16Gi` | | wrk-tclibs | `1000m` | `2000m` | `1Gi` | `2Gi` | | wrk-postproc | `2500m` | `~` | `2Gi` | `16Gi` | | wrk-cloud-cache | `1000m` | `4000m` | `1Gi` | `4Gi` | | wrk-cleanup-job | `1000m` | `2000m` | `1Gi` | `2Gi` | | wrk-health-job | `1000m` | `2000m` | `1Gi` | `2Gi` | | wrk-config-mng | `250m` | `1000m` | `128Mi` | `256Mi` | ### Authentication Authentication is achieved by leveraging [Authentication based on SubRequest Result](https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-subrequest-authentication/). This is natively supported by the Nginx Ingress Controller. In case a different ingress controller is used (e.g. ALB on AWS), additional Nginx Reverse Proxy is deployed in order to support the authentication mechanism. Authentication can be configured in the following ways: 1. Using external authentication service by specifying the `externalAuthUrl` in the configuration If an external authentication service is enabled, all header values from the incoming request will be forwarded to the external authentication service. The external authentication service needs to return following responses in order to support this authentication mechanism: - `HTTP 200`: authentication successful (ingress will forward the traffic to the backend service) - `HTTP 401` or `HTTP 403`: authentication failed 2. Using a simple authentication service deployed in the cluster Authentication service supports a simple Token check based on API path. The token needs to be included in the "Authorization" header with the "Token" prefix/scheme. ```bash curl -H "Authorization: Token " ``` The tokens are configured as secrets with the following behavior: | Secret (when custom secret name is used) | Secret (default secret name) | Type | Description | Used in deployments (Pods) | | :---- | :---- | :---- | :---- | :---- | | `` | `-secret-worker-api-token` | Optional | Token secret which contains token that is used to protect all endpoints with `/api/` prefix, e.g. file upload. | Auth | | `` | `-secret-worker-api-task-token` | Optional | Token secret which contains token that is used to protect `/api/tiscale/v1/task` endpoints. If left empty, the mentioned API is protected by `-secret-worker-api-token` | Auth | The authentication service won’t be deployed in the cluster if `externalAuthUrl` is defined: ```yaml # Example for enabling authentication worker: configuration: authentication: enabled: true externalAuthUrl: "" ``` ### Upload file for processing There are multiple ways to upload a file for analysis: - direct upload - upload from URL - upload of container image **Info: More information about the upload endpoints can be found in the [API Reference](../../API/UsageAPI.md#upload-a-file-for-processing).** #### Synchronous direct file upload Uploads a file for processing and waits until processing is complete, returning the full analysis report directly in the response. To modify synchronous API timeouts or connection limits, apply the appropriate annotations to the Ingress resource. #### Container Image Upload from Private Repository On container image upload, the image is directly pulled from the specified registry. To pull images from a private repository, the following conditions should be met: - Authentication: Valid credentials must be provided (e.g., a dockerconfigjson secret) for identity verification. - Trust/Encryption: If the registry uses a private Certificate Authority (CA), the relevant root certificates must be installed on the host or injected into the container runtime to establish a trusted connection. **Authentication credentials** Authentication credentials can be provided via **dockerconfigjson** secret, which supports credentials for multiple repositories. Like all other secrets, this secret can also have customized name and can be created from the Helm Chart. More information can be found in [Secrets](#secrets). If the secret is created with Helm Chart, the list with credentials needs to be provided in which each list item contains username, password and registry name. **Warning: There is a limitation on Helm Chart secret creation. For each registry, the credentials have to be provided because identity token is not yet supported.** If the secret is not created with Helm Chart, it needs to be created manually or already exist in the cluster (e.g. secret management tools like AWS Secrets Manager, HashiCorp Vault, etc.). Steps for manual creation: 1) Create config.json file with the following structure: ```json { "auths": { "https://some.repo.io/": { "auth": "BASE64_ENCODED_USER_PASS" }, "https://another.repo.io/": { "identitytoken": "abcdef..." }, "https://some-other.repo.io/": { "auth": "BASE64_ENCODED_USER_PASS" }, ... } } ``` ```sh BASE64_ENCODED_USER_PASS = echo -n "username:password" | base64 ``` 2) Create secret from the file: ```json kubectl create secret generic some-secret-name --from-file=.dockerconfigjson=config.json --type=kubernetes.io/dockerconfigjson ``` **Warning: Since reloader is not yet supported for Worker, manual restart of the receiver deployment is needed on secret change:** ``` kubectl rollout restart deployment/ ``` **Certificates** Certificates can be provided via **Opaque** secret. This secret has only one key **ca_bundle** which contains all the necessary certificates. Like all other secrets, this secret can also have customized name and can be created from the Helm Chart. More information can be found in [Secrets](#secrets). If the secret is created with Helm Chart, the list with certificates needs to be provided. To load the content of the file directly to the variable on upgrade/install, `--set-file` needs to be used: ```sh helm upgrade -i /detect-platform --set-file worker.secrets.caCerts.certificates[0]=/path/to/certificate/certificate1.pem --set-file worker.secrets.caCerts.certificates[1]=/path/to/certificate/certificate2.pem ``` On manual secret creation, the certificates need to be manually combined into one and secret needs to be created from the resulting file: ```sh cat certificate.pem certificate2.pem certificate3.pem > bundle.pem kubectl create secret generic cert-secret-name --from-file=ca_bundle=./path/to/file/bundle.pem ``` **Warning: Since reloader is not yet supported for Worker, manual restart of the receiver deployment is needed on secret change:** ``` kubectl rollout restart deployment/ ``` ## Connector S3 ### Introduction Connector S3 allows automatical retrieval of a large number of files from external S3 buckets. Connector S3 is composed of the following components: | Component | Description | | :---- | :---- | | `connector-s3` |Connector S3 service | ### Component Deployment Conditions For connectorS3 to work properly, `connectorS3.enabled: true` must be set in the values file and at least one input needs to be configured `connectorS3.configuration.inputs`. ### Application Secrets For proper connector S3 functionality, credentials for each input need to be configured. More information about Connector S3 input secret can be found in the [Connector S3 Secrets](./config-reference.md#connector-s3-secrets) section. Connector S3 input secret names can be customized and the secrets can be created with Helm Chart. More information about secret customization can be found [here](#secrets). ### Configuration Reference Configuration for Connector S3 can be found in the [Connector S3 Configuration Reference](config-reference.md#connector-s3). #### Default Resource Configuration | Component | CPU Request | CPU Limit | Memory Request | Memory Limit | | :---- | :---- | :---- | :---- | :---- | | connector-s3 | `4000m` | `~` | `2Gi` | `6Gi` | ## Spectra Detect Manager (SDM) ### Introduction Spectra Detect Manager (SDM) is a web-based management portal for Spectra Detect deployments. It provides centralized configuration, monitoring, and administration capabilities. The SDM deployment is packaged as an umbrella Helm chart (`detect-sdm`) that orchestrates the following components: | Component | Description | | :---- | :---- | | `sdm-portal` | Main web portal application | | `sdm-celery-worker` | Background task processor using Celery | | `sdm-data-change` | Data change notification service (conditional) | | `seaweedfs` | S3-compatible object storage (conditional) | | `clickhouse` | Analytics database for central logging (conditional) | | `postfix` | SMTP relay for email notifications | ### Component Deployment Conditions Not all SDM components are deployed by default. The umbrella chart uses the following conditions to determine which subcharts are installed: | Component | Deployment Condition | Description | | :---- | :---- | :---- | | `sdm-portal` | `sdmPortal.enabled: true` | Core portal, always deployed when SDM is enabled | | `sdm-celery-worker` | `sdmPortal.enabled: true` | Background task processor, always deployed when SDM is enabled | | `postfix` | `sdmPortal.enabled: true` | SMTP relay, always deployed when SDM is enabled | | `seaweedfs` | `sdmPortal.config.centralFileStorage.enabled: true` | Object storage, deployed only when central file storage is enabled | | `clickhouse` | `sdmPortal.config.centralLogging.enabled: true` | Analytics database, deployed only when central logging is enabled | | `sdm-data-change` | `sdmPortal.config.centralLogging.enabled: true` | Data change service, deployed only when central logging is enabled | ### Dependencies #### RabbitMQ and PostgreSQL RabbitMQ and PostgreSQL are required for SDM to function properly. PostgreSQL database is used for storing configuration and application data, and RabbitMQ is used for task queue management with Celery workers. These components can be externally provided or deployed as part of the same Detect Platform chart. View the [RabbitMQ](#rabbitmq-broker) and [PostgreSQL](#postgresql-server) documentation for more information. #### ClickHouse (Conditional) ClickHouse is deployed and used for central logging and analytics when `sdmPortal.config.centralLogging.enabled` is set to `true`. Enabling central logging also deploys the Data Change Service (DCS). ```yaml sdmPortal: config: centralLogging: enabled: true ``` **Info: View the [Clickhouse Configuration Reference](config-reference.md#clickhouse) for more information about the configuration options and the secrets configuration.** #### Data Change Service Secrets (Conditional) The Data Change Service (`sdm-data-change`) is deployed when central logging is enabled (`sdmPortal.config.centralLogging.enabled: true`). Uses the following credentials: - PostgreSQL - ClickHouse (optional) - Postfix (optional) - Spectra Intelligence (optional) **Info: View the [Data Change Service Configuration Reference](config-reference.md#data-change-service-dcs) for more information about the configuration options.** #### SeaweedFS (Conditional) SeaweedFS provides S3-compatible object storage for file management. It is deployed when central file storage is enabled (`sdmPortal.config.centralFileStorage.enabled: true`). Credentials are auto-generated (`-seaweedfs-secret`). **Note: Enabling central file storage requires central logging to also be enabled.** **Info: View the [SeaweedFS Configuration Reference](config-reference.md#seaweedfs) for more information about the configuration options.** #### Postfix Postfix provides SMTP relay capabilities for sending email notifications. **Info: View the [Postfix Configuration Reference](config-reference.md#postfix) for more information about the configuration options and the secrets configuration.** #### Loki and Prometheus (Optional) The SDM Portal optionally integrates with Loki for log aggregation and Prometheus for metrics. When the logging stack is deployed, the portal gets the connection details those components. **Warning: The Prometheus Configuration chart needs to be enabled to allow the SDM Portal to access the connection details. More information can be found [here](#prometheus-configuration).** ### Application Secrets Multiple SDM components create use secrets for application configuration and authentication. Some secrets are automatically generated by the charts and cannot be customized, but for most of the secrets used in SDM, the same applies as for other secrets: their names can be customized and they can be created from the Helm Chart. More information about secret customization can be found [here](#secrets). General information about the secrets that are customizable and used in SDM can be found here: - [SDM Portal Secrets](config-reference.md#sdm-portal-secrets). - [ClickHouse Secrets](config-reference.md#clickhouse-secrets). - [Postfix Secrets](config-reference.md#postfix-secrets). Since SDM is composed as an umbrella chart and contains multiple components, the custom secret names are defined on a global level. Configuration Reference for that can be found [here](config-reference.md#sdm-secrets-configuration-values). ### Configuration Reference Configuration for all SDM components can be found in the [SDM Configuration Reference](config-reference.md#spectra-detect-manager-sdm). #### Default Resource Configuration | Component | CPU Request | CPU Limit | Memory Request | Memory Limit | | :---- | :---- | :---- | :---- | :---- | | sdm-portal | `2000m` | `3000m` | `2Gi` | `4Gi` | | sdm-celery-worker | `1000m` | `3000m` | `2Gi` | `4Gi` | | seaweedfs | `1000m` | `3000m` | `2Gi` | `4Gi` | | clickhouse | `8` | `16` | `16Gi` | `32Gi` | | postfix | `500m` | `1000m` | `512Mi` | `1Gi` | | sdm-data-change | `500m` | `2000m` | `512Mi` | `2Gi` | ## Appendix ### Set Report Types - Report types can be added with the `--set-file` option by providing the name of the report type (added as a new key to `reportTypes`) and path to the file in which the filter is defined in JSON format - Report type name must match the one defined in the given file - Report types can be deleted by defining the name of the `report.type` and setting it to the value "" instead of adding the filepath - **Limitation**: max size of the report type file is 3MiB ### Set Report Types Example ```bash # Example of adding the 2 report types helm upgrade -i "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform -f custom_values.yaml \ --set-file worker.reportTypes.some_report_type=/some_report-type.json --set-file worker.reportTypes.extendedNoTags=/extendedNoTags.json # Example of adding the new report type and removing the existing report type helm upgrade -i "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform -f custom_values.yaml \ --set-file worker.reportTypes.some_report_type=/some_report-type.json \ --set-file worker.reportTypes.extendedNoTags="" \ ############### Example of the report type file content ############### { "name": "exampleReportType", "fields": { "info" : { "statistics" : true, "unpacking" : true, "package" : true, "file" : true, "identification" : true }, "story": true, "tags" : true, "indicators" : true, "classification" : true, "relationships" : true, "metadata" : true } } ``` ### Use Report Type Example Uploading a report type using the method above only makes the report type available to the system. To actually use the custom report type, you must configure it on the appropriate egress integration. For example, to use a custom report type with S3 storage: ```yaml # Specify the report type that should be applied to the Worker analysis report before storing it. Report types are results of filtering the full report. worker: configuration: reportS3: reportType: "custom_report_type" ``` ### Set YARA Rules - YARA rules can be added with the `--set-file` option by providing the name of the rule file (added as a new key to `yaraRules`) and path to the file in which the rule is defined - The rule file name must follow camelCase format - YARA rules can be deleted by defining the rule file name and setting it to the value "" instead of adding the filepath - **Limitation**: max size of YARA ruleset file is 45MiB ### Set YARA Rules Example ```bash # Example of adding yara rules helm upgrade -i "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-platform -f custom_values.yaml \ --set-file worker.yaraRules.rule1=/someYaraRule.yara \ --set-file worker.yaraRules.rule2=/other_yara_rule.yara # Example of adding the new yara rule and removing the existing yara rule helm upgrade -i "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform -f custom_values.yaml \ --set-file worker.yaraRules.rule1=/someYaraRule.yara \ --set-file worker.yaraRules.rule2="" ############### Example of the yara rule file content ############### rule ExampleRule : tc_detection malicious { meta: tc_detection_type = "Adware" tc_detection_name = "EXAMPLEYARA" tc_detection_factor = 5 strings: $1 = "example" $2 = "eeeeeee" condition: $1 or $2 } ``` ### Set Advanced Filter - Advanced filters can be added with the `--set-file` option by providing the name of the filter (added as a new key to `advancedFilters`) and path to the file in which the filter is defined in YAML format. Filter name must match the one defined in the given file - Advanced filters can be deleted by defining the name of the filter and setting it to the value "" instead of adding the filepath ```bash # Example of adding Advance Filter helm upgrade -i "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform -f custom_values.yaml \ --set-file worker.advancedFilters.filter1=/some_filter.yaml \ --set-file worker.advancedFilters.filter2=/other_filter.yaml # Example of adding Advance Filter helm upgrade -i "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform -f custom_values.yaml \ --set-file worker.advancedFilters.filter1=/some_filter.yaml \ --set-file worker.advancedFilters.filter2="" ``` Example of the filter file content: ```yaml name: some_filter description: Custom filter for Spectra Analyze integration scope: container type: filter_in condition: and: - range: info.file.size: gt: 50 lt: 20000 - one_of: classification.classification: - 3 - 2 ``` ### [Configuration Reference](config-reference.md) **Info: View the [Configuration Reference](config-reference.md) for more information about the configuration options.** ## Central Logging Central Logging is used to collect and display information about all the events happening on connected Worker. With this feature enabled, SDM home page shows information about processed files: classification, size and scan date, to name a few. Central logging is enabled if the following values are set to true: - `sdm.sdmPortal.config.centralLogging.enabled` - `worker.configuration.centralManager.queueLoggingEnabled` Events are stored in ClickHouse, which allows both components to efficiently exchange data and maintain a consistent state. ## Logging Logging is an optional sub-chart within the Detect Platform deployment. When enabled, it provides a complete data pipeline for managing logs across the environment. Contains the following components: - Alloy - Loki - Grafana ### Alloy Discovers and collects logs on the cluster. It can be configured to collect only logs from specific namespaces. **Info: View the [Alloy Configuration Reference](config-reference.md#alloy) for more information about the configuration options.** ### Grafana Provides a centralized dashboard for searching and visualizing logs in a streamlined, user-friendly interface. Global access is managed via Ingress resource, providing a secure and unified entry point for data exploration. There is a dashboard named "Kubernetes Logs" which shows logs from all the components of the Detect application in a specific namespace, apart from logging components and Prometheus. Default namespace is the one where Grafana is deployed. The dropdown menu allows for easy switching between individual namespaces or a global 'All Namespaces' view. **Info: View the [Grafana Configuration Reference](config-reference.md#grafana) for more information about the configuration options.** ### Loki Acts as the central repository for log storage, processing incoming data from Alloy and serving it to Grafana for seamless searching and analysis. **Info: View the [Loki Configuration Reference](config-reference.md#loki) for more information about the configuration options.** ## Scaling Scaling of the services is done in one of the following ways: - Scaling based on CPU usage - Scaling based on the number of messages waiting in the queue ### Scaling based on CPU usage Scaling based on CPU usage is implemented in the following way: - Users can provide `triggerCPUValue` which represents the percentage of the given CPU resources. Service will be scaled up when this threshold is reached and scaled down when CPU usage drops below the threshold. - CPU resources are defined with `resources.limits.cpu` value, which represents the maximum CPU value that can be given to the pod - **Default values**: - scaling enabled by default (except for SDM) - when CPU usage reaches 75%, the scaling is triggered - scaling delay is 30 seconds (scaling is triggered when scaling condition is met for 30 seconds) - one pod in every 30 seconds is created - number of maximum replicas is 8 - every 10 seconds the average CPU status is checked - **Worker services with CPU scaling**: - receiver - tclibs - **SDM services with CPU scaling**: - sdm-portal ### Scaling based on the number of messages waiting in the queue Scaling based on the number of messages waiting in the queue is implemented in the following way: - Scaling enabled by default (except for SDM) - User can provide `targetInputQueueSize` which represents the number of messages in the queue for which the scaling will start - different behavior for Worker and SDM components: - Worker services are scaled to 0 when the relevant queues are empty - Each worker service has defined at least two triggers: activation (0 -> 1, 1 -> 0) and scaling (1 -> N, N -> 1) - Unacknowledged messages are excluded from the calculation in scaling trigger, included for the calculation in activation trigger - Each scaled service has a different queue that is observed and scaled on - **Default values**: - when 10 or more messages are waiting in the queue for a longer period of time the scaling will be triggered - scaling delay is 15 seconds (scaling is triggered when scaling condition is met for 15 seconds) - one pod in every 30 seconds is created - number of minimum replicas is 0 - number of maximum replicas is 8 - every 10 seconds status of the queue is checked - **Worker services with Queue based scaling**: - processor: number of messages in the `tiscale.hagent_input` is used for scaling - processor-retry: number of messages in the `tiscale.hagent_retry` is used for scaling - postprocessor: number of messages in the `tiscale.hagent_result`/`tiscale.hagent_error` is used for scaling - preprocessor: number of messages in the `tiscale.preprocessing` is used for scaling (`tiscale.preprocessing_dispatcher` used only for activation) - preprocessor-unpacker: number of messages in the `tiscale.preprocessing_unpacker` is used for scaling - **SDM services with Queue based scaling**: - celery-worker: number of messages in the `tasks.default` is used for scaling ### Scaling configuration | Value | Description | |:--------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------| | `enabled` | enables/disables auto-scaling | | `maxReplicas` | maximum number of replicas that can be deployed when scaling in enabled | | `minReplicas` | minimum number of replicas that need to be deployed | | `pollingInterval` | interval to check each trigger (in seconds) | | `cooldownPeriod` | the period to wait after the last trigger reported active before scaling the resource back to 0, in seconds | | `scaleUp` | configuration values for scaling up | | `scaleUp.stabilizationWindow` | number of continuous seconds in which the scaling condition is met (when reached, scale up is started) | | `scaleUp.numberOfPods` | number of pods that can be scaled in the defined period | | `scaleUp.period` | interval in which the numberOfPods value is applied | | `scaleDown` | configuration values for scaling down | | `scaleDown.stabilizationWindow` | number of continuous seconds in which the scaling condition is not met (when reached, scale down is started) | | **CPU scaling** | | | `triggerCPUValue` | CPU value (in percentage), which will cause scaling when reached; percentage is taken from the resource.limits.cpu value; limits have to be set up | | **Queue scaling** | | | `targetInputQueueSize` | Number of waiting messages to trigger scaling on; unacknowledged messages are excluded | ## Secrets For all secrets used in the deployment, the following is applied: 1) Default secret names can be overridden with custom values to allow for easier integration with third-party secret managers. 2) All secrets must either be managed within the Helm Chart, created manually, or integrated via external secret management tools (e.g. AWS Secrets Manager, HashiCorp Vault, etc.). **Warning: WARNING** Use creation of secrets with Helm Charts for convenience/testing only. Do not store sensitive credentials in configuration files that are committed to Git. Consider using a dedicated secret management solution (e.g., Sealed Secrets, External Secrets, or Vault) for production environments. **Info: View the Configuration Reference for more information about the creating secrets with Helm Charts:** - [Worker Secret Configuration Values](./config-reference.md#worker-secret-configuration-values) - [Connector S3 Input Configuration Values](config-reference.md#connector-s3-application-configuration---file-storage-input-configuration) - [SDM Portal Secret Configuration Values](config-reference.md#sdm-portal-secret-configuration-values) - [ClickHouse Secret Configuration Values](config-reference.md#clickhouse-secret-configuration-values) - [Postfix Secret Configuration Values](config-reference.md#postfix-secret-configuration-values) - [SDM Custom Secret Names](config-reference.md#sdm-secrets-configuration-values) - [RabbitMQ Configuration Values](config-reference.md#rabbitmq-values) - [PostgreSQL Configuration Values](config-reference.md#postgresql-values) --- ## Spectra Detect Central SDM — Federated Manager Monitoring and YARA Policy Sync The Central Spectra Detect Manager (CSDM) is a management overlay that operates on top of deployed SDM instances. It provides a high-level, aggregated operational dashboard across all deployed SDM environments and ensures consistent security policies are maintained across the entire federated SDM deployment topology. ## Overview The current Spectra Detect Manager (SDM) implementation supports both Open Virtual Appliance (OVA) and monolith microservice deployment models, but cannot manage both simultaneously within a single SDM instance. This limitation arises because the configuration switches that govern the instance's deployment behavior are implemented as global parameters. The CSDM addresses this limitation by providing: - **Centralized monitoring**: Aggregated operational dashboard across all deployed SDM environments - **Policy synchronization**: Consistent YARA rule distribution across the entire federated SDM deployment topology - **Multi-deployment support**: Ability to manage both OVA and Kubernetes deployments from a single interface, supporting migration from legacy deployment models ### What Central SDM does not do The CSDM operates as a monitoring and YARA rule management plane. It does not replace the Spectra Detect Manager for day-to-day configuration tasks. The following operations remain on each local SDM: - Connector configuration (ingress and egress) - User management and access control - Quota settings - Classification change notifications - Central Configuration settings Local SDMs continue to serve as the only integration point for Worker and Spectra Analyze appliances. The CSDM is not a direct integration endpoint. ## Requirements - All local SDMs registered with the Central SDM must run version 5.7.2 or later - OpenID Connect (OIDC) authentication is strongly recommended for Single Sign-On (SSO) functionality ## Architecture The CSDM uses a centralized approach where a single SDM instance manages all deployments, providing a unified view over the entire system while respecting the specifics of different deployment types. The CSDM transparently aggregates groups from different environments. ### Design approach The CSDM architecture is based on an API-driven model where the CSDM acts as a lightweight orchestrator rather than a complete data repository. It relies on APIs to communicate with local SDMs, sending requests to relevant instances when information is needed or changes must be made. Each local SDM remains the true source of its own data. All communication from the CSDM to local SDMs is initiated by the CSDM. This design simplifies network configuration and prevents the need for complex, direct connections between components. ### Local Spectra Detect Manager registration Local Spectra Detect Manager (SDM) instances are registered with the Central SDM through configuration. This registration specifies the necessary parameters for the SDM's integration into the broader architecture. ### Data ingestion and M2M communication The Central SDM collects operational data and telemetry from the registered local SDMs using the standard SDM API. Authentication for this Machine-to-Machine (M2M) communication is secured through a token-based mechanism. The CSDM operates solely as a centralized monitoring and management plane only. ### Communication patterns #### API requests The CSDM sends API requests to local SDMs when configuration changes or information retrieval is needed. For Kubernetes deployments, a Manager component serves as a proxy between the CSDM and individual appliances. The appliance ID is included as part of the request, and the Manager forwards the request to the specific hub or worker. #### Appliance events Events from appliances are collected by local SDMs and stored in persistent storage. The CSDM pulls events from local SDMs on a regular polling schedule, reversing the typical communication direction for event handling. ### YARA rule synchronization #### Centralized rule authority The CSDM maintains the master copy and serves as the single source of truth for all YARA rules. The CSDM is responsible for aggregating and resolving rule conflicts from all connected Detect and Analyze appliances. #### Distribution mechanism The CSDM pushes YARA rules to local SDMs. For Kubernetes deployments, local SDMs serve as repositories from which workers pull rules. This maintains the standard worker behavior while ensuring centralized rule management. #### Upgrade path and initial data seeding An existing cluster or set of clusters can be switched to the CSDM. During this transition, the existing single SDM is temporarily designated as the initial source, and its active YARA rules are imported into the CSDM repository. After import, all local SDMs are required to use the YARA rules managed by the CSDM. #### Request forwarding Spectra Analyze appliances continue to direct requests to their respective local SDMs. However, the local SDMs are configured to forward all YARA-related requests from both Spectra Analyze appliances and Workers to the CSDM for centralized processing. Sample-specific operations like egress configuration and retro hunting remain local. Worker synchronization is also forwarded to CSDM. Synchronization status is resolved by local SDM by fetching the rules versions from CSDM and comparing them with Workers' statuses. ## Configuration ### Central SDM The Central SDM (CSDM) pulls operational telemetry from local SDM instances. Local SDM discovery information is configured in the `multiRegion` section of the CSDM's ConfigMap. Local SDMs are enumerated here, and the overall multi-region polling feature is activated by setting the `enabled` flag to true. ```yaml multiRegion: enabled: true initialYaraSyncPullId: vm-1 sdmList: - id: vm-1 name: "OVA First" url: https://10.200.1.1 type: "ova" - id: vm-2 name: "OVA Second" url: https://10.200.2.2/ type: "ova" - id: k8s-2 name: "AWS Kubernetes Third" url: "https://third.detect-dev.reversinglabs.com/" type: "kubernetes" - id: k8s-1 name: "AWS Kubernetes Fourth" url: "https://fourth.detect-dev.reversinglabs.com/" type: "kubernetes" ``` #### Configuration parameters - **enabled**: Activates or deactivates Central SDM multi-region polling - **initialYaraSyncPullId**: Specifies which local SDM environment serves as the initial source of YARA rules. This must match one of the `id` values in the `sdmList` - **sdmList**: Defines all local SDMs that the Central SDM monitors - **id**: Internal identifier for the SDM (used by APIs and UI routing) - **name**: Display name shown in the Central SDM UI - **url**: Base URL of the local SDM - **type**: Deployment type (`ova` or `kubernetes`) #### Initial YARA rule synchronization When configuring Central SDM, exactly one environment is designated as the initial YARA rule source via the `initialYaraSyncPullId` parameter. This environment's Spectra Analyze appliances should contain the YARA rulesets that will be distributed to all other environments. All other Spectra Analyze appliances must start without existing rulesets. They will receive their initial rulesets through synchronization from the designated source environment. After the initial synchronization is complete, the Central SDM maintains a consistent ruleset across all environments. ### Local SDM The local Spectra Detect Manager (SDM) generally operates independently of the CSDM. The exception to this independence is the YARA rule synchronization process. The local SDM assumes a simple consumer role, exclusively fetching the master YARA rule set from the central CSDM. This behavior is configured exclusively on the CSDM side via an API operation. Refer to the Central SDM API documentation for endpoint details. This design decision ensures compatibility with deployment environments, such as OVA-based clusters, where configuration by a ConfigMap is not available. Local SDMs are registered at the Central SDM. No separate configuration on each local SDM is required beyond standard YARA synchronization with Spectra Analyze and workers. ## Using Central SDM ### Authentication Authentication for the Central SDM is configured to be consistent with the approach used for the local Spectra Detect Manager (SDM) instances. It is strongly recommended that OpenID Connect (OIDC) be deployed as the primary authentication mechanism. Using OIDC enables Single Sign-On (SSO), allowing users to seamlessly transition between the CSDM interface and local SDM management consoles without requiring re-authentication. For machine-to-machine communication, local SDMs are authenticated with the CSDM using token-based authentication. Workers are authenticated with their local SDM instances as needed. ### Landing page The UI of the CSDM is similar to the regular SDM UI, but with reduced functionality. Most of it is located on the single, initial screen. On the left side, it shows a list of the configured SDMs together with their type icon (VM/Kubernetes), name, and status. ![image](./images/CSDM.png) SDM statuses are: - **Unknown**: Before status is known - **Online**: Everything is operating normally - **Degraded**: Some hubs or workers are down - **Error**: Some component reported an error - **Offline**: Secondary unavailable - **Maintenance**: Maintenance or restart in progress - **Unauthorized**: Invalid primary authentication configuration ### Appliances tab The Appliances tab on the SDM-CM interface largely mirrors the core status presentation found in a standalone local SDM. However, it incorporates CSDM specific controls which act as a centralized gateway to manage individual instances. Added controls: - **Manage button**: Launches the UI of the specific local SDM in a new browser tab. This is an authenticated launch (preferably via SSO) for direct configuration and troubleshooting. ### YARA synchronization The overview screen provides a granular status display, detailing the current sync state for each appliance. Possible synchronization states include: - **InSync**: The local SDM has successfully fetched and applied the latest rule set - **OutOfSync**: The local SDM is using an older version of the rules - **Error**: The last attempted synchronization failed - **Unavailable**: The CSDM cannot establish communication with the local SDM endpoint - **PendingNew**: A new rule set is available on the CSDM-CM, awaiting local SDM retrieval - **Disabled**: Synchronization for this local SDM has been administratively deactivated - **NoRules**: The local SDM has not yet received any rules from the CSDM ## Upgrading to Central SDM Upgrading from a single existing local SDM to Central SDM is the supported migration path. The process involves designating an existing SDM as the initial YARA rule source and importing its active rules into the Central SDM. ### Migration considerations - All SDMs in a Central SDM deployment share the same YARA rules - The upgrade requires a maintenance window during which YARA synchronization is paused and rules are migrated - Environments with multiple pre-existing SDMs that have independent rulesets require manual alignment before migration ### Migration procedure 1. Prepare the existing SDM that will serve as the initial rules source: - Place the SDM into maintenance mode - Stop YARA synchronization between Spectra Analyze and the SDM - Export active YARA rules from this SDM 2. Import rules into the Central SDM 3. Configure the local SDM to consume rules from the Central SDM 4. Re-enable synchronization (Central SDM to local SDM to workers) 5. Add additional SDMs to the Central SDM configuration. These will automatically adopt the centralized ruleset ## Limitations and considerations ### Functional scope The Central SDM UI does not include the following features, which remain available only on each local Spectra Detect Manager instance: - Central Configuration page - Connector settings (ingress and egress configuration) - Classification change notifications - Local user and access management ### Operational considerations - The Central SDM does not push configuration changes to local SDMs. It serves as a monitoring and YARA rule authority only - If you need to decommission the Central SDM or remove an instance from it, coordinate this change with support to avoid interruptions in rule management - All connected SDMs must share a single, unified YARA ruleset managed by the Central SDM --- ## Spectra Detect Deployment — OVA, AMI, Kubernetes EKS, and Multi-Region Spectra Detect appliances are available as OVA, AMI, or container images. Use your platform’s standard deployment process, and refer to the guides below for detailed instructions. Additionally, you can deploy Spectra Detect using Kubernetes. Kubernetes is an open source platform installed on Linux servers according to the manufacturer's instructions, and supports several different Linux distributions, including current versions of Ubuntu 24.04 LTS and Debian 12. ## Deployment Guides --- ## Spectra Detect Multi-Region Deployment — High Availability and Data Residency A multi-region deployment distributes Spectra Detect across two or more geographic locations to achieve high availability, disaster recovery, and data residency compliance. Files are analyzed in the region where they are submitted, and a centralized SDM cluster provides unified management across all regions. ## Architecture ![Spectra Detect multi-region deployment](../images/spectra-detect-multi-region.png) A single load balancer sits in front of both regions and receives all incoming traffic from file sources (email gateways, web proxies, endpoint agents, file storage, and cloud buckets). It routes each request to the active region based on health, geography, or routing policy. Within each region, an SDM node and a Hub node run as an active/standby pair. Region A runs the active SDM and the active Hub; Region B runs the standby SDM and standby Hub, kept in sync via state replication. Workers run as a cluster in each region and process files submitted by the local Hub. A worker in Region B handles overflow or failover from Region A when needed. Analysis results flow from the Worker clusters through the Hubs and out to the configured output destinations — SOAR, SIEM, EDR, sandbox, and threat intelligence platforms — regardless of which region performed the analysis. ## When to use multi-region deployment Use a multi-region deployment when you need: - **Data residency** — files must be analyzed in a specific jurisdiction and must not leave that region. - **Low latency** — file sources in different geographies connect to the nearest Hub rather than routing across continents. - **High availability** — a regional outage does not stop file analysis in other regions. - **Disaster recovery** — the SDM failover ensures management continuity even if one data center is lost. ## Components | Component | Role in multi-region | |---|---| | SDM load balancer | Routes SDM API/UI traffic to the active SDM node; fails over to standby across regions | | SDM cluster (active + standby) | Central control plane; state-synced between nodes | | Hub load balancer | Distributes file submission traffic to regional Hub clusters by geography or policy | | Regional Hub cluster (active + standby) | Ingests files and distributes them to Workers in the same region | | Regional Worker cluster | Performs file analysis locally; scales independently per region | ## Prerequisites Before deploying across multiple regions: - An SDM instance must be reachable from all Hub clusters (consider network peering or a transit gateway between regions). - Each regional Hub and Worker cluster must be deployed and registered with SDM before enabling cross-region routing. - TLS certificates must cover all regional endpoints, or a wildcard/SAN certificate must be issued for the load balancer and regional hostnames. **Load balancer requirements by platform:** | Platform | SDM load balancer | Hub load balancer | |---|---|---| | On-premises data center | Customer-provided hardware or software load balancer (for example, F5 BIG-IP, HAProxy, or NGINX) with health-check and failover support | Customer-provided load balancer with geographic or policy-based routing between data center sites | | AWS | AWS Global Accelerator or Application Load Balancer (ALB) | AWS Global Accelerator with endpoint groups per region, or Route 53 latency-based or geolocation routing | | Microsoft Azure | Azure Front Door or Azure Load Balancer with Traffic Manager | Azure Front Door with origin groups per region, or Azure Traffic Manager with geographic routing | | Google Cloud | Google Cloud Load Balancing (global external HTTP(S) load balancer) | Google Cloud Load Balancing with backend services per region, or Cloud DNS with geolocation routing policies | ## Deployment steps ### 1. Deploy the SDM cluster Deploy SDM in your primary region following the appropriate [deployment guide](./index.md) for your platform (OVA, AMI, or Kubernetes). Configure the standby SDM node in a second region and enable state synchronization between them. Place a load balancer (or DNS failover) in front of both SDM nodes so that API and UI clients use a single hostname regardless of which node is active. ### 2. Deploy regional Hub and Worker clusters For each region: 1. Deploy a Hub cluster (active + standby) following the appropriate [deployment guide](./index.md) for your platform. 2. Deploy a Worker pool in the same region and register it with the local Hub. 3. Register the Hub cluster with SDM using the SDM management API or UI. Repeat for each region. Each region should have its own Hub cluster and Worker pool that operate independently. ### 3. Configure the Hub load balancer Configure your global load balancer to route incoming file submission traffic to the correct regional Hub cluster. Common routing strategies: - **Geographic routing** — route requests to the nearest region based on the client's location. - **Latency-based routing** — route to the region with the lowest measured latency from the client. - **Failover routing** — designate a primary region and fail over to a secondary if the primary becomes unhealthy. ### 4. Verify cross-region operation After deployment: 1. Submit a file from each region and confirm it is analyzed by Workers in the correct regional cluster. 2. Check the SDM dashboard and confirm all regional Hubs and Workers appear as connected and healthy. 3. Test SDM failover by stopping the active SDM node and confirming the standby takes over without interrupting Hub connectivity. 4. Confirm YARA rules deployed through SDM propagate to Workers in all regions. ## Considerations **YARA rule synchronization** — SDM pushes YARA rule updates to all registered Workers across all regions. Ensure network connectivity between SDM and every regional Worker cluster. **License** — each Worker node consumes a license seat regardless of region. Contact your ReversingLabs account team to confirm your license covers the total Worker count across all regions. **Analysis results** — by default, each regional Hub stores results locally. Configure S3 output or webhook forwarding if you need results aggregated centrally. **SDM availability** — if the SDM cluster is unreachable, Hubs and Workers continue processing files using their last-known configuration. YARA rule updates and license checks are paused until SDM connectivity is restored. --- ## Spectra Detect File Analysis — Worker API, Reports, and Spectra Core Results Spectra Detect Worker analyzes files submitted via the [Worker API](../API/UsageAPI.md) and produces a detailed analysis report for every file using the built-in Spectra Core static analysis engine. Analysis reports can be retrieved in several ways, depending on the Worker configuration. It is also possible to control the contents of the report to an extent. ## Retrieving Analysis Reports There are two ways to get the file analysis report(s): 1. The [Get information about a processing task](../API/UsageAPI.md#get-information-about-a-processing-task-taskinfo) endpoint. Sending a GET request to the endpoint with the task ID returns the analysis report in the response. 2. Saved on one of the configured integrations. - **S3** - for hosted deployments, this is the only supported integration - Microsoft services (Azure Storage, SharePoint, OneDrive) - file shares (NFS, SMB) - Splunk - callback server ## Adding Custom Data to the Report Users can also save any custom data in the analysis report by submitting it in the file upload request. The `custom_data` field accepts any user-defined data as a JSON-encoded payload. This data is included in all file analysis reports (Worker API, Callback, AWS S3, Azure Data Lake and Splunk, if enabled) in its original, unchanged form. The `custom_data` field will not be returned in the [Get information about a processing task](../API/UsageAPI.md#get-information-about-a-processing-task-taskinfo) endpoint response if the file has not been processed yet. Users should avoid using `request_id` as a key in their `custom_data`, as that value is used internally by the appliance. **Example - Submitting a file with the custom_data parameter to store user-defined information in the report** ```bash curl https://tiscale-worker-01/api/tiscale/v1/upload -H 'Authorization: Token 94a269285acbcc4b37a0ad335d221fab804a1d26' -F file=@Classification.pdf -F 'custom_data={"file_source":{"uploader":"malware_analyst", "origin":"example"}}' ``` ## Customizing Analysis Reports There are several different ways of customizing an analysis report: 1. through report configuration 2. through report **types** 3. through report **views** These methods are not mutually exclusive and are applied in the order above (configuration first, then report type, then report view). For example, to even be present for later filtering/transforming, **strings** found in a file must be included in the report. **Report types** are results of *filtering* the **full** report. In other words, fields can be included or excluded as required. On the other hand, **report views** are results of *transforming* parts of the report, such as field names or the structure of the report. Historically, views could also be used to just filter out certain fields without any transformations, and this functionality has been maintained for backward compatibility. However, filtering-only views should be replaced by their equivalent report types as they are much faster. As previously mentioned, filtering and transforming actions are not mutually exclusive. You can filter out some fields (using a report type), and then perform a transformation on what remains (using a report view). However, not all report views are compatible with all report types. This is because some report views expect certain fields to be present. ### Report Types Report types are JSON configuration files with the following format: ```json5 { "name": "string", "exclude_fields": true, "fields": { "example_field": false, "another_example": { "example_subfield": false, "another_subfield": false } } } ``` Some default options: - `small` - Contains only the classification of the file, and some information about the file. - `extended_small` - Contains information about file classification, information about the file, the `story` field, `tags` and `interesting_strings`. - `medium` - This is the **default report** that’s served when there are no additional query parameters (in other words, it’s not necessary to specifically request this report, as it’s sent by default). - It is equivalent to the previous "summary" report with some small differences: - each subreport contains an `index` and `parent` field - if `metadata.application.capabilities` is 0, then this field is not present in the report - Changes in this report: - excludes the entire `relationships` section - excludes certain fields under the `info` section, such as `warnings` and `errors` - many `metadata` fields are not present such as those related to certificates - there are no strings, no story and no tags - `large` - Includes every single field present in the analysis report. It is equivalent to the previous "full" report (`?full=true`). **Note: You can upload custom report types to the appliance through **Central Configuration > Egress Integrations**, where you can also manage other settings for each integration. Custom report types are visible for all integrations, regardless of which section was used to upload it.** ------ Report types that replace report views with the same name: - `classification` - This report returns only the classification of the file, story and info. It has no metadata except the `attack` field. - `classification_tags` - Same as the classification view, with the addition of Spectra Core tags. - `extended` - Compared to the default (medium) type, contains:all metadata, relationships, tagsthe `story` fieldunder `info`, contains statistics and unpacking information - `mobile_detections` - Contains mobile-related metadata, as well as classification and story. - `mobile_detections_v2` - Contains more narrowly defined mobile metadata, with exclusive focus on Android. Also contains classification and story. - `short_cert` - Contains certificate and signature-related metadata, as well as indicators and some classification info. The `name` of the report type is the string you’ll refer to when calling the [Get information about a processing task](../API/UsageAPI.md#get-information-about-a-processing-task-taskinfo) endpoint (or the one passed to the relevant configuration command). For example, if the name of your report type is `my-custom-report-type`, you would include it in the query parameters as follows: `?report_type=my-custom-report-type`. The `exclude_fields` field defines the behavior of report filtering. This is an optional field and is `false` by default. This means that, by default, the report fields under `fields` will be **included** (you explicitly say which fields you **want**). Conversely, if this value is set to `true`, then the report fields under `fields` will be **excluded** (you explicitly say which fields you **don’t want**). The `fields` nested dictionary contains the fields that are either included or excluded (depending on the value of `exclude_fields`). If a subfield is set to a boolean value (`true`/`false`), then the inclusion/exclusion applies to **that section and all sections under it**. For example, `small.json`: ``` { "name": "small", "fields": { "info" : { "file": true, "identification": true }, "classification" : true } } ``` In this configuration, we’re explicitly **including** fields (`exclude_fields` was not set, so it’s `false` by default). Setting individual fields to `true` will make them (and their subfields) appear in the final report. In other words, the only sections that will be in the final report are the entire `classification` section and the `file` and `identification` fields from the `info` section. Everything else will not be present. Or, `exclude-example.json`: ``` { "name": "exclude-example", "exclude_fields": true, "fields": { "relationships": false, "info": { "statistics": false, "binary_layer": false, } } } ``` In this configuration, the entire `relationships` section is excluded, as well as `statistics` and `binary_layer` from the `info` section. Everything else will be present in the report. #### Limitations - `info.file` cannot be excluded and is always present. - **Lists cannot be partially filtered.** If you include a list field (such as `indicators` or `interesting_strings`), all items in that list are included with all their primitive fields. You cannot selectively include or exclude individual fields within list items. For example, if you want to include `indicators` but only show the `priority` field: ```json { "name": "custom-report", "fields": { "indicators": { "priority": true } } } ``` This will **not** work as expected. The entire `indicators` list will be included with all fields (`priority`, `category`, `description`, `reasons`, etc.). This limitation applies to all list types in the report schema, including: - `indicators` (struct-list) - `interesting_strings` (struct-list) - `strings` (struct-list) - `scan_results` (struct-list) - `tags` (list) - `story` (struct-list) You can include or exclude these lists as a whole, but you cannot selectively filter individual fields within list items. This applies regardless of whether you use include mode (`exclude_fields: false`) or exclude mode (`exclude_fields: true`). - **Primitive fields cannot be excluded if they share a level with an included field**. Primitive types (*string*, *number*, *boolean*, *null*) on the same level as an included field, or above it, will always be included. Only complex objects (structs) and lists can be excluded at those levels. Example structure: ```json { "classification": { "propagated": false, "classification": 3, "factor": 2, "result": "Win32.Trojan.Generic", "rca_factor": 7, "scan_results": [ { "name": "TitaniumCore RHA1", "classification": 3 } ], "propagation_source": { "sha1": "a94a8fe5ccb19ba61c4c0873d391e987982fbbd3" } } } ``` If you include only `result` (the threat name): - `propagated`, `classification`, `factor`, `rca_factor` are included (primitives on the same level as `result`) - `scan_results` and `propagation_source` are excluded (complex types not explicitly included) Filtering result: ```json { "classification": { "propagated": false, "classification": 3, "factor": 2, "result": "Win32.Trojan.Generic", "rca_factor": 7 } } ``` ### Report Views Views are transformations of the JSON analysis output produced by the Worker. For example, views can be used to change the names of some sections in the analysis report. There are also deprecated views that allow filtering fields in or out, but this functionality is covered by report types (see above). The following views are present by default (deprecated views are excluded): - `classification_top_container_only` - Returns a report view equivalent to the `classification` report **type** (see above), but for the top-level container (parent file). - `flat` - "Flattens" the JSON structure. Without flattening: ```json "tc_report": [ { "info": { "file": { "file_type": "Binary", "file_subtype": "Archive", "file_name": "archive.zip", "file_path": "archive.zip", "size": 20324816, "entropy": 7.9999789976332245, ``` With flattening: ```json "tc_report": [ { "info_file_entropy": 7.9999789976, "info_file_file_name": "archive.zip", "info_file_file_path": "archive.zip", "info_file_file_subtype": "Archive", "info_file_file_type": "Binary", ``` - `flat-one` - Returns the `flat` report, but only for the parent file. - `no_goodware` - Returns a short version of the report for the top-level container, and any children files that are suspicious or malicious (goodware files are filtered out). This view is not compatible with split reports. - `no_email_indicator_reasons` - Strips potential PII (personally identifiable information) from some fields in analysis reports for email messages, and replaces it with a placeholder string. - `splunk-mod-v1` - Transforms the report so that it’s better suited for indexing by Splunk. The changes are as follows: - if `classification` is 0 or 1, `factor` becomes `confidence` - if `classification` is 2 or 3, `factor` becomes `severity` - a `string_status` field is added with the overall classification (UNKNOWN, GOODWARE, SUSPICIOUS, MALICIOUS) - scanner `name` becomes `reason` - scanner `result` becomes `threat` Views can generally be applied to both split (available in self-hosted deployments) and non-split reports. If none of these views satisfy your use case, contact [ReversingLabs Support](mailto:support@reversinglabs.com) to get help with building a new custom view. ## Interpreting the report After sending files to be processed, you will receive a link to a JSON report. It contains a `tc_report` field, which looks something like this: ```json "tc_report": [ { "info": { "file": { "file_type": "Text", "file_subtype": "Shell", "file_name": "test.sh", "file_path": "test.sh", "size": 35, "entropy": 3.7287244452691413, "hashes": [] } }, "classification": { "propagated": false, "classification": 0, "factor": 0, "scan_results": [ {} ], "rca_factor": 0 } } ] ``` ### High-level overview The key information here is the classification value (`tc_report[].classification.classification`), which will be a number from 0 to 3: | `classification` | description | |------------------|------------------| | 0 | no threats found | | 1 | goodware | | 2 | suspicious | | 3 | malicious | ### More information For more information, use the `tc_report[].classification.rca_factor` field. The higher its value, the more dangerous the threat, except for files that weren’t classified (their `classification` is 0). In that case, `rca_factor` will be 0 and will not signal trustworthiness. For even more information on why a file was given a certain classification, look at the `scan_results`. This field contains the individual scanners which processed the file (`name`), as well as their reason for classifying a file a given way (`result`). The following table maps the `classification` value to the old "trust factor" and "threat level" values, and the new "RCA factor" value which replaces them. It also provides a mapping to a color-coded severity value, and provides general commentary with examples regarding the origin of any given classification. ### Risk tolerance profiles Based on your risk tolerance, you can set up the downstream systems (for example, a SIEM/SOAR) to act on different values in the classification field. Here are two example risk tolerance profiles - adapt them to your own use case: For a broader overview of classification, `rca_factor`, and how they affect priority, see the [Classification](/General/AnalysisAndClassification/Classification) guide. #### Low risk tolerance - receive alerts on all possible threats (maximum number of matches) - act on both suspicious and malicious verdicts ```python classification = report['tc_report'][0]['classification']['classification'] if classification >= 2: alert_SOC() ``` #### High risk tolerance - receive alerts only on highly impactful threats ```python classification = report['tc_report'][0]['classification']['classification'] risk_score = report['tc_report'][0]['classification']['rca_factor'] if classification == 3 and risk_score >= 7: alert_SOC() ``` ## Spectra Core Spectra Detect Worker relies on the built-in Spectra Core static analysis engine to classify files and produce the analysis report. The file classification system can produce the following classification states: *goodware*, *suspicious*, *malicious*. With this classification system, any file found to contain a malicious file will be considered malicious itself if classification propagation has been enabled in the configuration. In the default configuration, propagation is enabled. Multiple technologies are used for file classification, such as: format identification (malware packers), signatures (byte pattern matches), file structure validation (format exploits), extracted file hierarchy, file similarity (RHA1), certificates, machine learning (for Windows executables), heuristics (scripts and fileless malware) and YARA rules. These are shipped with the static analysis engine, and their coverage varies based on threat and file format type. ## Classifying Files with Cloud-Enabled Spectra Core Spectra Core can be connected to Spectra Intelligence to use file reputation data. This data is not based solely on antivirus scanning results, but on the interpretation of the accuracy of those results by ReversingLabs, as well as on the analyst-provided (manual) classification overrides. Note that the only information Spectra Core submits to the Spectra Intelligence cloud is the file hash. Connecting Spectra Core to the cloud will add threat reputation to the `scan_results` in the report, for example: ```json { "ignored": false, "type": "av", "classification": 0, "factor": 0, "name": "", "rca_factor": 0 } ``` To connect Spectra Core to Spectra Intelligence, in the Manager, go to *Central Configuration > Spectra Intelligence*. ### How Spectra Intelligence Enhances Spectra Core Classification **Threat Naming Accuracy** When classifying files, Spectra Core takes all engines listed in the analysis report (Spectra Intelligence included) into consideration. Based on their responses, it selects the technology that provides the most accurate threat naming. More specific methods that identify the malware family more accurately are given precedence. Generic and heuristic classification are picked last, and only if there is no better-named classification response. Spectra Intelligence generally returns specific threat names, and it will be selected as authoritative if a better option is not available. It can also enhance Spectra Core classification results. For instance, Spectra Core machine learning can classify malware only with heuristic, non-named classification. If Spectra Core finds ransomware via machine learning, the threat name will appear as *Win32.Ransomware.Heuristic*. However, if Spectra Core is connected to Spectra Intelligence, the cloud response can change the threat name to something better-defined, such as *Win32.Ransomware.GandCrab*. This helps users understand exactly which malware family they are dealing with, as opposed to just the threat type (ransomware). **Whitelisting and Goodware Overrides** When not connected to Spectra Intelligence, Spectra Core can classify files as goodware either based on digital certificates that were used to sign the files, or via graylisting - a system that can declare certain file types as goodware based on the lack of code detected within them. When connected to Spectra Intelligence, whitelisting can be expanded based on file reputation and origin information. As a result, the number of unknown files (files without classification) will be significantly reduced. Users also get an insight into the trustworthiness of whitelisted files measured through trust factor values. If classification propagation is enabled on Spectra Core, a whitelisted file can still be classified as suspicious or malicious if any of its extracted files is classified as suspicious or malicious. Goodware overrides is a feature designed to prevent this. When enabled, it ensures that any files extracted from a parent and whitelisted by certificate, source, or user override can no longer be classified as malicious or suspicious. Spectra Core automatically calculates the trust factor of the certificates before applying goodware overrides, and does not use certificates with the trust factor lower than the user-configured goodware override factor. With goodware overrides enabled, classification is suppressed on any files extracted from whitelisted containers. In this case, whitelisting done by either Spectra Intelligence or certificates will be the final classification outcome. Spectra Core will still report all malicious files it finds, but they won’t be displayed as the final file classification. This feature allows for more advanced heuristic classifications that have a chance of catching supply chain attacks. As those rules tend to be noisy, they can be suppressed by using this feature. The user can still see all engine classification results, and can use them to proactively hunt for possible supply chain attacks. Note that the goodware overrides will not apply if the extracted file is found by itself (for example, if an extracted file is rescanned without its container). ### Mapping Spectra Intelligence and Spectra Core Classification States When Spectra Core is connected to Spectra Intelligence, it uses a combination of two classification systems - Spectra Core file classification and Malware Presence (MWP). While their *malicious* and *suspicious* classification states translate well from one to another, MWP *known* to Spectra Core *goodware* does not. By default, Spectra Core converts all MWP *known* states to *goodware*. This can be problematic when false negative scans happen in the Spectra Intelligence cloud, as the cloud would declare a file as KNOWN, but in reality, the file would be undiscovered malware. Such files generally have a low trust factor value. To resolve those issues, users can rescan files classified as non-malicious to confirm whether they are false negatives, or configure Spectra Core to map Spectra Intelligence *known* to *goodware* based on the trust factor. Users can configure the MWP Goodware Factor value, which defines the threshold for automatically converting MWP *known* to Spectra Core *goodware*. When this value is configured, instead of converting *known* to *goodware* in all cases, Spectra Core will only convert it when a file has a trust factor value equal to or lower than the configured one. By default, the value is configured to 5 (convert all), and the recommended value is 2. In this case, if a file classified as *known* in the cloud has a trust factor greater than 2, Spectra Core will not consider that file as *goodware*. It will be considered unclassified (with no threats found), and its cloud classification will not be present in the list of scanners in the Spectra Core analysis report. ## Spectra Detect Decision Matrix The following section explains how to interpret the classification data provided by Spectra Detect. The intent is to maximize the effectiveness of malicious classifications, while reducing the negative impact false positive detections might have. Start the decision-making process by looking at the top-level file, found first in the Spectra Detect report. Perform the following checks in order. 1. **No Threats Found classification** If the value of `tc_report.classification.classification` is "0" (no threats found), Spectra Detect detected no threats and the file is not present in ReversingLabs Cloud (or in the T1000 database). The sample could receive a classification at a later date, or it can be uploaded for analysis to the Spectra Intelligence cloud. After cloud classification, the sample may be marked as 3 (Malicious), 2 (Suspicious) or 1 (Known/Goodware). ReversingLabs reserves the "Known/Goodware" classification only for samples that the classification algorithm deems as trustworthy. If a sample remains unclassified, analysis did not find malicious intent at that time, but the sample and its metadata are not trustworthy enough to be declared "Known/Goodware." 2. **Known classification** If the value of `tc_report.classification.classification` is 1 (known), the file has been analyzed and the analysis did not find any known threats. To determine if the file is goodware, trusted and clean, perform the following checks: 1. If the value of `tc_report.classification.factor` is 0 or 1, the file is goodware and it comes from a highly reputable source. 2. If the value of `tc_report.classification.factor` is 2 or 3, the file is clean and it comes from known public sources usually unrelated to malware infections. 3. If the value of `tc_report.classification.factor` is 4 or 5, the file is likely clean and it comes from known public sources, some of which have been known to carry malware in the past. Files with this factor can change to other classifications over time, or their factor can improve when they are found in better sources. 3. **Suspicious classification** If the value of `tc_report.classification.classification` is 2 (suspicious), the file has been analyzed and the analysis found a possibility that the file is a new type of threat. This classification category is reserved for static analysis and cloud reputation heuristics, and it can lead to false positives. Depending on the risk aversion profile, two approaches are advised: 1. High risk tolerance - suspicious classifications are allowed as heuristics trigger often on files. This most commonly happens in the case where the files are collected in a corrupted or truncated state before analysis. 2. Low risk tolerance - suspicious classifications are not allowed. However, filtering on specific reasons for suspicious classifications is still available via the first element in the `tc_report.classification.scan_results[0].name` list. 4. **Malicious classification** If the value of `tc_report.classification.classification` is 3 (malicious), the file has been analyzed and recognized as a known malware threat. Depending on the risk aversion profile, two approaches are advised: 1. High risk tolerance - malicious files are not allowed, but PUA (potentially unwanted applications) are. Lower risk malware, PUA, have `tc_report.classification.factor` set to 1. In case PUA are allowed, an additional filter that blocks files with a factor greater than 1 is advised. 2. Low risk tolerance - malicious files are not allowed regardless of classification reason. ## Classification Propagation Spectra Detect unpacks files during analysis, so it is possible to have a file that is classified based on its content. Any file that contains a malicious or suspicious file is also considered malicious or suspicious because of its content. ### Propagated Classification Suppression In some cases, unpacked files might contain files that are misclassified. These false positives are propagated to the top, and it may appear that the entire archive is malicious. Suppression of these classifications is possible, and is safe, under the following conditions: 1. The classification is caused by propagation. When this happens, the optional field `tc_report.classification.propagation_source` exists. 2. Either of the following or all of the following: - The top-level file has been found in a trusted source. Find the scanner named `av` in the `tc_report.classification.scan_results` scanner list. If it exists, and its classification is 1 (known), check its factor value. If the factor value is 0 or 1, the file is goodware and it comes from a highly reputable source. - The top-level file is signed by a trusted party. Find the scanner named "TitaniumCore Certificate Lists" in the `tc_report.classification.scan_results` scanner list. If it exists and its classification is 1 (known), the file is goodware regardless of its factor value. ### Propagation Suppression Example The following false positive scenario illustrates how the suppression logic is applied: 1. A file is considered whitelisted because it is signed by a trusted digital certificate. 2. It has been classified as known and highly trusted in ReversingLabs Cloud, and has no positive antivirus detections in ReversingLabs Cloud. 3. However, during extraction, one or more malicious files were detected inside the file, and one or more malicious detections have been declared as a false positive. The classification of the file that has been submitted to Spectra Detect is indicated in the top-level section. Since it contains the `propagation_source` field, that indicates the top-level file is considered malicious because it contains at least one malicious file. The SHA1 value points to this file, which is the origin of the final top-level classification. The scanners in the `scan_results` list enumerate all the factors that contributed to the final classification. For example, they might include: 1. Resulting classification from propagation: Classification is malicious ( `classification: 3` ) with `factor: 2` 2. Certificate whitelisting for the top-level file: Classification is goodware ( `classification: 1` ) with `factor: 0` 3. Cloud response for the top-level file: Classification is goodware ( `classification: 1` ) with `factor: 0` The suppression algorithm can be applied to this top-level file, as it is not only signed with a whitelisted certificate, but is also considered known and highly trusted in ReversingLabs Cloud. ## Deep Cloud Analysis Pipeline Diagram When Deep Cloud Analysis is enabled on Spectra Detect, the appliance uses the following pipeline to analyze file submissions. ```mermaid flowchart TD %% Compact layout with all-rectangle nodes; only YES, NO, SKIP, SCAN uppercase S[START] --> A[Is the hash in user_data hash block list?] A -->|YES| X1[SKIP] A -->|NO| B[Is the hash in the global hash block list?] B -->|YES| X2[SKIP] B -->|NO| C[Is full scan enabled?] C -->|YES| Y1[SCAN] C -->|NO| D[Is this an unpacked file?] D -->|YES| E[Is the file type blocked?] E -->|YES| X3[SKIP] E -->|NO| F[CONTINUE] D -->|NO| F F[Was the file previously scanned?] F -->|NO| Y2[SCAN] F -->|YES| G[Is rescanning enabled?] G -->|NO| X4[SKIP] G -->|YES| H[Is the previous scan result older than the configured rescan interval?] H -->|YES| Y2 H -->|NO| X5[SKIP] ``` --- ## Spectra Detect Dashboard — Manager UI, Navigation, and Status Indicators This short introductory section is intended to help with understanding the basic layout of the user interface, terminology and visual indicators that are used on the Manager and in the rest of this User Guide. ## Global Header Bar At the top of the Manager interface is the global header bar, containing the most commonly used options and the main appliance menu used to access all sections of the Manager. - **Quota**: Usage-based quota insights for all Spectra Intelligence accounts used by the connected appliances. Clicking the triangle icon expands the header and displays the limit and the license renewal dates for each account. Quota limit statuses are color indicated. Additionally, this menu contains the option to contact ReversingLabs Support. - **Dashboard**: The dashboard displays statistics related to the amount and type of files that have been submitted and processed on the appliance within a specified time range. - **Central Configuration**: The [Central Configuration Manager](../Config/ApplianceConfiguration.md) allows users to modify configuration settings on Spectra Detect appliances directly from the Manager interface. The Central Configuration feature makes it easier to configure appliances remotely, and to ensure that the settings are consistent and correct across multiple appliances. The appliances must first be connected and authorized on the Manager instance. **Spectra Analyze appliances can be configured using the Spectra Detect Manager APIs.** - **Administration**: Allows users to access and configure [Spectra Detect Manager Settings](../Admin/ManagerSettings.md), [YARA Synchronization](../Admin/YARASync.md), [Redundancy](../Admin/Redundancy.md), and [Email Alerting](../Admin/EmailAlerting.md). - **Help**: Contains an option to access Manager API Documentation and the online product documentation. - **Notifications**: Contains a bell icon that shows unread notifications for cloud classification changes. Clicking the `See all notifications` redirects users to the [Notifications](./Notifications.md) page, where they can view and manage all notifications. - **User menu**: Shows the username of the current user, contains link to user details, and the option to Log Out from the appliance. - **Integrations Status Indicators**: Contains arrow labels providing information on which file input sources and file/report output sources are currently configured on the connected Hub groups. ## Integrations Indicators The **Integrations Indicators** at the top-right of the interface contain `File Ingress Connectors` and `File Egress Connectors` providing information on which file input sources ([Connectors](../Config/AnalysisInput.md)) and file/report output sources ([Egress Integrations](../Config/ApplianceConfiguration.md#egress-integrations)) are currently configured on the connected Hub groups. Green icons indicate that the item has an existing configuration, not necessarily that the configuration is correct, and the service is connected, whereas grey icons indicate that the item does not have an existing configuration. Hub groups with configured Connectors are listed below the Connector, and outside of parentheses. ## Central Logging The Manager dashboard has two modes: central logging *enabled*/*disabled*. If central logging is enabled (*Administration > Spectra Detect Manager > Central Logging*), users can access the **Analytics** tab which shows various statistics for the processed files before showing appliance status. If it's not enabled, the default tab is **Appliance Management** showing the status of the Manager and all connected appliances. When central logging is enabled, the Analytics tab on the dashboard shows a detailed breakdown of analyzed files according to classification state, file type, malware family, the total size of all processed files, and more. This classification data and error logs can be exported to CSV by clicking the `Export Classification Data` button at the top of the page. Note that some processed files in these .csv files appear with slightly changed names, e.g. `-` become `[-]`. This is a measure to ensure sanitized input. ![](../images/c1000-dashboard-logging.png) Exported classification data CSV files contain the following information: sample (container), classification, rca_factor, malware_platform, malware_type, malware_family, filename, file_size (bytes), file_type_group, file_type, file_subtype, identification, processed_at, hostname. ## Detections Overview The Detections Overview table is a list of files analyzed on connected Workers. It updates in 15 second intervals, and shows sample classification, file name, size, file type, threat name, scan date, AV detections, SHA1 and SHA256 hashes, source, source tag, and detailed analysis. All columns, apart from File, can be enabled or disabled using the gear icon menu in the top right of the table section. The table can be sorted by clicking the column headers, and filtered using the `Show Table Filters` button which reveals text field filters above the table columns. Results can additionally be filtered by the time they were analyzed using the drop-down menu at the top of the table. ### Source and Source Tag Columns The **Source** and **Source Tag** columns provide file origin tracking and end-to-end traceability for analyzed samples. These fields help SOC teams identify submission patterns, pinpoint submitters, and reduce investigation time. - **Source**: Displays the submission method or integration used to submit the file (e.g., `API`, `ICAP`, `S3`, `IMAP`). This field is automatically populated based on how the file was submitted to Spectra Detect. - **Source Tag**: Displays custom metadata that identifies the line of business, department, or organizational unit from which the file originated. This field can be set via: - The `source_tag` field in the `custom_data` parameter for API submissions - The `X-System-Tag` header for ICAP submissions - Automatically by connectors based on the submission source Both columns are filterable, allowing users to aggregate and analyze submissions by source. For example, SOC managers can view how many files were submitted from specific lines of business over a given time period, or identify which departments are submitting the most files. For more information on configuring the source tag, see the [Usage API documentation](../API/UsageAPI.md) and [ICAP Server configuration](../Config/AnalysisInput.md#icap-server). Clicking the `Showing LIVE Results` button stops the table from automatically updating, which can be useful while inspecting a specific set of results. Automatic updating is also paused when the user navigates from the first page of results. Cloud analysis results do not affect the dashboard classification. However, if a newer cloud classification differs from the existing dashboard classification, a red indicator is displayed in the *Classification* column to signal that reprocessing is recommended. The *AV Detections* column in the Detections Overview table displays results from the Deep Cloud Analysis (Multi-Scanning) Service, combining Worker Static Analysis and Spectra Intelligence analysis. It can be expanded to show the names of AV scanners and any detected threats, reflecting the analysis conducted by the Spectra Intelligence, where files are sent to AV scanners in the Cloud, with the results then exposed on the Manager dashboard. To enable the Deep Cloud Analysis (Multi-Scanning) Service, navigate to *Administration > Spectra Detect Manager > Dashboard Configuration*, and select the `Enable Multi-Scanning` checkbox. When the Multi-Scanning is enabled, Workers upload samples to the Cloud only if the sample does not already exist in the Cloud, and passes the filtering criteria: up to 2GB in size. If the sample already exists in the Cloud, the Manager will monitor for any changes in the data and update the Manager dashboard accordingly. Read more details in the Multi-Scanning section of the [Spectra Detect Manager Settings](../Admin/ManagerSettings.md) chapter. The `RESCAN` button sends a request to the Cloud to check for any updates associated with the submitted samples, and updates the information accordingly. The `See All Scans` button opens a new page with all available Spectra Intelligence and AV Detections information. ### Product integration with Spectra Analyze The `Detailed Analysis` column in the Detections Overview table allows the users to import the sample analyzed on the Spectra Detect platform into Spectra Analyze for a deeper insight into the analyzed file. For easier filtering, files imported to the Spectra Analyze appliance using this feature will automatically be tagged with the `spectra_detect` tag. For this integration to be enabled, the Manager must be connected to at least one instance of Spectra Analyze 8.1 or higher, and central logging (*Administration > Spectra Detect Manager > Dashboard Configuration > Central Logging*) must be enabled. In addition, the sample must be stored either in an S3 bucket or on the Manager itself (Central Storage). If both of these features are enabled, Manager central storage takes priority (Spectra Analyze will download files from the Manager). When Spectra Analyze integration is enabled and files are uploaded, the Sample Details link is available in the `Detailed Analysis` column, allowing users to directly open the file analysis page on the configured Spectra Analyze instance. This link remains available for the same files even after disabling the integration and takes priority over the Central File Storage and AWS S3 import links when multiple options are enabled. ## Processing Timeline The Processing Timeline section of the dashboard shows a graph of uploaded, processed and post-processed samples, and also the number of samples that failed to analyze. To retrieve a list of files that failed to analyze on the connected Spectra Detect appliances in the last 90 days, use the `Export Errors / Hashes` button. The exported CSV file contains the following information: - `host_uuid` - `hostname` - `time` - `event_type` - `task_id` - `sample` - `container` The `host_uuid` value is set automatically when the Worker connects to the Manager, and obtainable using the `conf_centralmanager` command on the Worker. Note that exported error logs might contain double entries for some errors. For example, if a file processing task fails, causing a failed report upload to S3, this is counted as two errors, despite being one event. ## Malware Types / Malware Family Count The Malware Types and Malware Family Count charts show the analyzed samples categorized by Malware Type and Malware Family Count, respectively. Malware Type is presented as a percentage in a pie chart while Malware Families are displayed as a bar chart indicating the sample count per family. ## Appliance Management ![](../images/c1000-dashboard.png) System information about the Manager instance and connected appliances can be found on the **Appliance Management** tab and is updated every 5 minutes. The *Status* column indicates whether the appliance is online, offline, unlicensed, in error state, or unauthorized. If YARA ruleset synchronization is enabled (*Administration > Spectra Detect Manager > General*) and on at least one connected appliance that supports it, a *YARA* column will show the current YARA ruleset synchronization status for each appliance. Possible YARA synchronization statuses are *In Sync*, *Not In Sync*, *Unknown*, *Please Update* and *Please Set To HTTPS*. See the [YARA Synchronization](../Admin/YARASync.md) section for more details. ### Worker pause and unpause For Spectra Detect Workers, an **Actions** column provides pause and unpause controls. Pausing a Worker allows it to complete currently processing samples but prevents it from accepting new ones. This is useful for planned maintenance, upgrades, or temporarily reducing processing capacity. To resume normal operation, click the unpause button. The Worker will immediately begin accepting new samples again. The Appliance Management page can be configured to display up to 100 appliances per page, and also filtered using the `Show Table Filters` button at the top of the list. This displays filter input fields in a row above the appliances table. Each table column has its own filter input field, allowing simultaneous filtering by multiple criteria. Keep in mind that some actions (like configuration changes) will result in a system restart. Depending on the type of appliance, the process of restarting and reloading the configuration might take some time. Spectra Detect Worker appliances generally take longer to restart. The following table describes common management actions and their consequences. | ACTION | APPLIANCE RESTART | MANAGER RESTART | |------------------------------------------------|-------------------|------------------| | Update the Manager instance | NO | YES | | Modify settings on the Manager instance | NO | YES | | Connect an appliance to the Manager | NO | NO | | Authorize an appliance | NO | NO | | Update a connected appliance | YES | NO | | Modify settings on the Appliance Status page | YES | NO | | Disconnect an appliance from the Manager | NO | NO | ## Connecting Appliances to the Manager **Note: Adding the same appliance to multiple Managers is not supported. It can lead to misconfigurations and conflicts. Always remove the appliance from one Manager before adding it to another.** To add an appliance, click `Add new appliance` on the Appliance Management tab on the Dashboard. In the *Add new appliance* dialog, choose the appliance type, then enter a name and an URL for the new appliance. **All appliance URLs must use the HTTPS protocol.** Note that the Manager does not validate SSL certificates automatically, so users must ensure their certificates are valid prior to connecting appliances. The *SNMP community* field is required for the appliance to properly communicate with the Manager, and for the Manager to display accurate status information on the dashboard page. Note that the SNMP community string set here must match the string previously configured on the appliance itself. If the selected appliance type is *TiScale Hub*, an additional field called *Load Balancer password* becomes available in the dialog. If the password is provided here, the appliance status page will display a tab with Load Balancer (HAProxy) statistics. Note that the password must be previously configured directly on the Hub, and on all Workers connected to that Hub. Clicking `Add` for Spectra Analyze appliances will redirect to the appliance login page, where the appliance must be authorized in order to successfully connect to the Manager. If authorization does not complete successfully, the appliances will be added to the Manager and displayed on the dashboard with *Unauthorized* status. They can be authorized at any moment from the [Appliance Status](#appliance-status-page) page. Workers and Hubs are immediately added to the Manager after clicking `Add`, without the authorization step. ## Appliance Status Page Apart from allowing access to *Update*, *Edit*, *Authorize*, and *Remove* options, the appliance status page provides detailed information about the system resources and health of the appliance. If there is a problem with retrieving the status information from the appliance, a warning message is displayed at the top of the appliance status page. In this case, users should check whether the SNMP community string is properly configured for the appliance on the Manager, and on the appliance itself. ![](../images/c1000-tabs.png) The information on the appliance status page is divided into tabs and refreshed every 5 minutes. The appliance type - Spectra Analyze, Spectra Detect Worker, Spectra Detect Hub - determines which tabs will be visible. | TAB | SUPPORTED ON | DESCRIPTION | |------------------|---------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | **System** | All appliance types | Displays the appliance type, status, version, name, description, and uptime. | | **CPU** | All appliance types | Displays the number of cores on the appliance, the average load over the last minute for each individual core and for all cores. | | **Storage** | All appliance types | Shows the total storage size (in GB), the amount of used storage (in GB and percentage) and allocation units for each partition on the appliance. | | **Network** | All appliance types | Provides an overview of network devices on the appliance, with information about their type, operational and administrative status (up/down), physical address, bandwidth (in Mb/s or Gb/s), and the total amount of traffic received and sent. | | **Queues** | All appliance types except Spectra Detect Hub | If supported by the appliance type, displays the state of queues on the appliance. Queues are used to communicate between various background services. The state of all queues should be “running”, and each queue should have at least one consumer (a service connected to the queue). | | **Processing** | Worker only | If supported by the appliance type, provides statistics on the amount and size of files submitted, processed, and unpacked on the selected Worker in each of the predefined time intervals. The *Queue AVG* column indicates the average number of files in the incoming queue for each of the predefined time intervals. | | **Load Balancer**| Hub only | If supported by the appliance type, provides real-time information on the status of the HAProxy service used for load balancing on the Hub. The data is updated every 10 seconds. Hovering over column names in the tables provides tooltips that clarify what each column refers to. This tab is visible only if the load balancer (HAProxy) password is provided in the appliance configuration dialog. | | **Metrics** | Worker only | If supported by the appliance type, displays additional file processing and post-processing statistics. The statistics track the following events: files successfully uploaded to Worker and sent for analysis; success and failure events for file processing on the Worker; success and failure events for uploading parent files to S3; success and failure events for uploading analysis reports to Splunk and Callback server (if those report uploading options are configured on Worker). The statistics are collected for the same predefined time intervals as in the *Processing* tab, and preserved for the maximum duration of 1 day (24 hours). The statistics are collected using HTTP calls to the Worker service API, but the SNMP community string must be set for this tab to be visible. Counts in each interval automatically adjust to indicate only the events that occurred within the exact interval, while all events exceeding it are removed from the count. Note that the count precision may be impacted if the system is under a heavier load, but it should improve within 2-3 minutes. Additionally, extracted files are not individually counted in S3 upload events - only their parent files are. This may cause discrepancies between the count of files processed on Worker versus files uploaded to S3. | ### RabbitMQ Queues Based on the appliance type and configuration, the following RabbitMQ queues may be available. #### Spectra Detect Worker | Queue | Description | | --- | ------------- | | `tiscale.preprocessing_dispatcher` | Waits for AV analyses to complete, then routes samples to `hagent_input`, retry, or preprocessing based on thresholds and Deep Cloud Analysis settings. | | `tiscale.hagent_result` | Stores successful analysis results from `ts-worker` for postprocessing. | | `tiscale.hagent_url` | Handles submit-url and submit-docker-image requests: downloads samples and routes them to `hagent_input`, retry, or preprocessing based on thresholds and Deep Cloud Analysis settings. | | `tiscale.hagent_error` | Stores failed analysis results from `ts-worker` for postprocessing. | | `tiscale.preprocessing` | Manages subscription and upload of samples for AV scanning when Deep Cloud Analysis is enabled. | | `tiscale.preprocessing_unpacker` | Unpacks samples before preprocessing when Deep Cloud Analysis is configured to scan child files. | | `tiscale.hagent_input` | Receives samples for analysis. Routes to `hagent_result` on success or `hagent_retry` on failure. Preprocessing occurs first if Deep Cloud Analysis is enabled. | | `tiscale.hagent_retry` | Handles retry attempts for failed analyses. Routes to `hagent_result` on success or `hagent_error` on final failure. | ### Additional Options The "Upload SSL Certificate" button allows the users to apply new HTTPS certificates to Workers, Hub, or Manager. The Actions > Download logs button downloads a support archive containing relevant system logs from all connected appliances. Appliance logs can also be downloaded using the [Download support logs endpoint](../API/ServiceAPI.md#download-support-logs). #### Spectra Detect Hub For Hubs that are configured as a Hub group (redundancy cluster), the appliance status page contains additional information above the tabs and a button to promote the secondary Hub into primary. The link *Redundant with other Hub instance* allows users to quickly access the other Hub instance in the cluster, and view its status page. ## Editing and Removing Appliances To edit an appliance, click on its row in the Dashboard to access the appliance status page. Click `Edit` to open the *Configure appliance* dialog. Here the name, host name, URL, and the SNMP (Simple Network Management Protocol) community string of the appliance can be modified. The HTTPS protocol is mandatory for the appliance URL. For Spectra Detect Hub appliances, it's also possible to provide the HAProxy password to enable HAProxy monitoring. Click `Save` to save changes, or `Cancel` to return to the appliance status page without saving. To remove an appliance, click `Remove` on the appliance status page. Confirm the removal in the popup dialog that appears at the top of the page. To safely remove an appliance, always use the *Remove* option on the appliance status page when the appliance is online. Attempting to remove or replace an appliance by changing its URL, or removing it while it is offline will result in an error. Appliances periodically check whether they are still connected to the Manager. If an appliance is removed improperly, the Manager will detect it, and the appliance will be automatically removed from the Manager. ## Authorizing Spectra Analyze While Workers and Hubs are authorized automatically, Spectra Analyze appliances are not. In that case, the *Status* column in the dashboard shows an "Unauthorized" message. The `Authorization` button is visible on the status page of an unauthorized appliance, and it redirects to the authorization page on Spectra Analyze. ### Enabling Appliances Search on Spectra Analyze Spectra Analyze appliances connected and authorized to the same Manager instance can be used to perform an Appliances Search for samples. The Appliances Search feature looks for samples on all connected and authorized Spectra Analyze appliances, and provides links to the results on each appliance from Spectra Analyze Uploads page. Users can search for samples by file name and sample hash. Multi-hash search is supported, and different types of hashes (for example, MD5 and SHA1) can be submitted in one query. A notification message will appear if an appliance is not reachable or if the search cannot be performed on an appliance. To enable the Appliances Search feature on a Spectra Analyze appliance, access the *Administration > Configuration > Spectra Detect Manager* dialog and select the *Enable Appliances Search* checkbox. --- ## Spectra Detect Notifications — Classification Alerts and Delivery Rules Users can access the notifications page from the header by clicking the notifications icon, which will display unread notifications, providing a quick overview of alerts that require attention. Clicking the **See all notifications** link redirects users to the notifications page, where they can view all notifications. A table on the notifications page displays all notifications, separated in columns by *Type*, *Time*, and *Notification*. The *Type* column indicates the type of notification. The *Time* column displays the timestamp of the notification, indicating when the event occurred. The *Notification* column provides a brief description of the event, such as a classification change from unknown (no threats found) to malicious. Filtering options are available to help users quickly find relevant information. Notifications can be [filtered by period](#filter-by-period), allowing users to view alerts from the last hour, day, week, month, or all time. Users can also [filter by read status](#filter-by-read-status), distinguishing between read and unread notifications, [filter by notification type](#filter-by-notification-type), including Cloud Classification Changes and Classification Detection, or [filter by classification](#filter-by-classification), narrowing the results to cloud classification changes where a sample was marked as unknown (no threats found), malicious, suspicious, or goodware. Clicking on the hash value within the alert redirects users to the **Dashboard > Analytics > Detections Overview** table, providing additional context and information about the sample. The **Mark All as Read** button allows users to clear unread notifications by marking them as read. This can be used to quickly clear the notification list and focus on new alerts as they arrive. ## Filter by Period The filter by period option allows users to view notifications from the last hour, day, week, month, or all time. This can be used to quickly identify recent alerts and track changes in the classification status. ## Filter by Read Status The filter by read status option allows users to distinguish between read and unread notifications. This can be used to quickly identify new alerts that require attention or to review previously read notifications for additional context. ## Filter by Notification Type The filter by notification type option allows users to distinguish notifications from Cloud Classification Changes and Classification Detection. ## Filter by Classification The filter by classification option allows users to narrow results based on the classification of the sample, including samples marked as Unknown (No Threats Found), Malicious, Suspicious, or Goodware. ## Notification Settings The notification settings page allows users to configure and manage custom notification rules for tracking cloud classification changes. The page provides an overview of existing notifications, displaying their *Name*, *Type*, associated *Alert* type, *Description*, and *Action*. Users can navigate through the list using pagination controls and adjust the number of rows displayed per page. If no notifications are configured, the table remains empty. A button labeled **Add Notification** in the upper-right corner allows users to create new notification rules. ### Adding a Notification To add a new notification, users must first specify a *name*, *description*, and select a *notification type*. When choosing cloud classification changes, users can define the conditions by selecting the original classification (*cloud classification changes from*) and the new classification (*cloud classification changes to*) that will be used to trigger the notifications. Users can also choose the delivery method, including *E-mail*, *Splunk*, or *Syslog*, to ensure alerts are sent through the appropriate channels. - *E-mail* delivery method requires users to enter the recipients' *email addresses* and select the desired *notification frequency*. - *Splunk* delivery method requires users to enter the Splunk *protocol* (`http` or `https`), *host*, *port*, and *token*. - *Syslog* delivery method requires users to enter the Syslog *server*, *port*, *protocol* (`UDP` or `TCP`), *tag*, and *priority* level. ### Manage Profiles The Profile section allows users to manage their personal information and credentials. It includes fields for *First Name*, *Last Name*, and *Email Address*. Users can update their password by entering a new password, repeating the new password, and providing their current password for verification. These options ensure users can securely manage their account settings within the notification system. --- ## Spectra Detect YARA Hunting — Custom Rules, Modules, and Worker Sync ## Classifying Files with YARA Rules ## Using YARA with Spectra Detect Worker Default YARA rulesets on the appliance are automatically installed with the Spectra Core static analysis engine. With every engine update, these rulesets are updated as well. The rulesets cannot be saved to the Spectra Intelligence cloud or modified (edited, disabled, or deleted) in any way by any type of user. Additionally, ReversingLabs publishes open source YARA rules in [a public GitHub repository](https://github.com/reversinglabs/reversinglabs-yara-rules). These rules can be freely downloaded and imported into any Worker. In addition to default YARA rulesets, the Worker can use custom rulesets created by users. This is available by pulling rulesets from other Spectra Analyze and Worker appliances using [the YARA Sync feature](../Admin/YARASync.md) on Spectra Detect Manager. ### Rulesets and restrictions ReversingLabs products support the following YARA modules: - PE - ELF - Math - Hash - Time - Dotnet **Note: "Import" and "include" statements are not supported.** Save custom YARA rulesets as files with the *.yara* extension. Naming restrictions: - YARA ruleset names must be between 3 and 48 characters. - The underscore ( _ ) should be used instead of spaces, and any other special characters should be avoided. Ruleset names should only use numbers (0-9) and a-z/A-Z letters. **Tip: For more information on writing YARA rulesets, consult one of the following sources:** - ReversingLabs publishes guidance for using YARA on the official blog. See the blog posts ["Level up your YARA game"](https://blog.reversinglabs.com/blog/level-up-your-yara-game) , ["Writing detailed YARA rules for malware detection"](https://www.reversinglabs.com/blog/writing-detailed-yara-rules-for-malware-detection) and ["Five Uses of YARA"](https://blog.reversinglabs.com/blog/five-uses-of-yara) to learn more. - The official YARA documentation offers detailed advice on [how to write YARA rules](https://yara.readthedocs.io/en/stable/writingrules.html). - Use Spectra Core rulesets present on the Spectra Analyze appliance as reference. ## Synchronizing YARA Rulesets via Spectra Detect Manager In order to synchronize YARA rulesets, the Worker appliance must be connected to a Manager, and YARA syncing must be enabled on that Manager. The worker-c1000 section briefly explains how to connect a Worker to a Manager. If the Worker is connected to a Manager which has YARA synchronization enabled, rulesets from the Worker will be automatically synchronized with other appliances, and vice-versa. Likewise, when YARA synchronization is disabled on a Manager that Worker is connected to, it will be automatically disabled on the Worker as well. When YARA synchronization is enabled on a Manager, the Worker will poll it for new and updated rulesets once per minute. The YARA Sync page on the Manager will display a table of all appliances connected to the Manager and their YARA ruleset synchronization status. Appliances can show one of the following statuses: - **In Sync** - The rulesets on the connected appliance match the rulesets on the Manager. - **Not In Sync** - The connected appliance doesn’t have the newest YARA rulesets. - **Unknown** - The connected appliance doesn’t have YARA synchronization enabled, or is unreachable. - **Please Update** - The connected appliance needs to be updated to a newer version before it can show the YARA synchronization status. - **Please Set To HTTPS** - The appliance is connected to the Manager using the unsupported HTTP protocol. The appliance URL must be updated to `https://` in the *Configure appliance* dialog on the Manager. Workers will poll the Manager for rule changes every minute. Spectra Analyze appliances will push new rules to the Manager as soon as they are created, and pull new rules every 5 minutes. **Example case: One Spectra Analyze appliance and one Worker attached to a Manager with YARA Synchronization enabled** Any YARA ruleset change made on Spectra Analyze will take up to 5 minutes to be synchronized to the Manager. Once the change reaches the Manager, it will take up to 1 minute for the change to be synchronized to the Worker. In total, it will take up to 6 minutes for a change to be synchronized from a Spectra Analyze appliance to a Worker. Appliances that are *Not In Sync* can be manually synchronized at any time by clicking the *Start YARA Sync* button in the far right column of the table. Rulesets created on Spectra Analyze appliances before YARA synchronization was enabled won’t synchronize to the Manager until the user changes their status or modifies them in any way. Rules present on the Manager, however, will synchronize to newly connected Spectra Analyze appliances regardless of when they were created. Apart from new rulesets, changes in existing rulesets will be synchronized as well. If a ruleset is disabled or deleted on one appliance, its status will be distributed to other appliances. In case of Workers, disabled rulesets will be removed until they are re-enabled on another appliance. When enabled again, those rulesets will be synchronized to the Worker as if they have been newly created. This means that the Worker only contains enabled (synchronizable) rulesets at all times. ## Troubleshooting YARA Issues on the Worker 1. From the Worker appliance status page on the Manager, disconnect the Worker by clicking the *Remove* button. 2. Access the dashboard page and click the *Add new appliance* button. 3. In the *Add new appliance* dialog that opens, select *Spectra Detect Worker* as the appliance type, and fill in the configuration fields with the data of the previously disconnected Worker instance. 4. Click *Submit* to connect the Worker instance to the Manager again. If the process completes successfully, the YARA Sync page on the Manager should display the status of the Worker instance. --- ## Spectra Detect Usage — Analysis, Dashboards, and YARA Management --- ## Spectra Detect Analysis Input — Connector Configuration for Email and S3 **Connectors** allow users to automatically retrieve a large number of files from external sources such as email or S3 buckets. Users can configure this service from the Appliance status page. Select a connected Hub, then select the **Connectors** button. **Note: The Hub must belong to a Hub group with at least one Worker.** If a connector is disabled or if it has not been previously configured on the appliance, the dialog contains only the **Enable connector** button. Click the button to start configuring the connector. Never use the same folder for both input and output of files. ## Starting the Connector When the configuration is complete, click **Start connector** at the bottom of the page. This will initiate the connector service on the appliance. After starting, the connector service connects to the configured user account, automatically retrieves files from it, and submits them for analysis on the appliance. Each file retrieved via the connector has a set of User Tags automatically assigned to it. Those User Tags are based on the file metadata, and can contain information about the file source, the last modification time in the original location, file permissions, email subject, recipient and sender addresses, and more. If advanced options are not enabled, the connector service will not perform any additional actions on the files retrieved from the server after the appliance finishes analyzing them. The users can see the analysis results for each file on its Sample Details page. After providing the required information, click **Test connection** to verify that the appliance can access the configured service storage. When the button is clicked, the appliance attempts to connect and mount the service storage. To remove all configured settings for the current service storage, click **Remove item**. To add another service storage, click **Add item**. Up to five units of storage can be added this way. If there are already five units of storage connected to the appliance, at least one must be removed by clicking **Remove item** before adding a new one. Note that for S3 connector, this limit is 20. ## Pausing and Disabling the Connector While the connector service is active, the *Start connector* button changes into **Pause connector**. Clicking this button temporarily halts the connector service. The connector service records the last state and is able to resume scanning when **Start connector** is clicked again. While the connector is running, it is possible to modify its configuration and save it by clicking **Save changes** without having to pause or disable the connector. If the connector service is active during a scheduled or manually executed Purge action, the system will automatically stop the service before performing the Purge action, and start it after the Purge action is complete. To disable the entire connector service on the appliance, click **Disable connector** at the bottom of the page. When the connector is disabled, it will not be possible to reconfigure, start, or pause it until the service is enabled again. Note that the current connector configuration will be preserved and restored when the service is enabled again. Likewise, all files that have been retrieved and analyzed by Spectra Analyze will remain on the appliance. All files retrieved from the server and analyzed on the appliance are accessible to Spectra Analyze users from the Submissions page. They are distinguished from other files by a unique username specific for each connector. Spectra Detect Workers will follow the default data retention policy. All processed files are deleted immediately after processing. If a file is queued but not processed within 9 hours, the processing task will be canceled (and the file deleted) but the record of the unsuccessful task will still be present in the database for 24 hours. All file processing results are retained until deleted by the user, or for 9 hours after processing (whichever comes first). ## IMAP - MS Exchange - AbuseBox Connector The IMAP - MS Exchange AbuseBox connector allows connecting to a Microsoft Exchange server and analyzing retrieved emails on the appliance. ### Requirements - IMAP must be enabled on the Exchange server. - A new user account must be configured on the mail server and its credentials provided to the connector in the configuration dialog. - A dedicated email folder must be created in the Exchange user account, and its name provided to the connector in the configuration dialog. All emails forwarded to that folder are collected by the connector and automatically sent to the appliance for analysis. When the analysis is complete, email samples with detected threats will get classified as malicious and, if the automatic message filing option is enabled, moved to the specified *Malware* folder. Emails with no detected malicious content do not get classified. They can optionally be moved to the specified *Unknown* folder on the configured Exchange user account. To improve performance and minimize processing delays on Spectra Analyze, each email sample will get analyzed and classified only once. When the *Automatic message filing* option is enabled, each email sample is moved only once, based on its first available classification. Because of that, it is recommended to enable classification propagation and allow retrieving [Spectra Intelligence](/SpectraIntelligence/) classification information during sample analysis instead of after. Administrators can enable these two options in the **Administration ‣ Configuration ‣ Spectra Detect Processing Settings** dialog. This will improve classification of emails with malicious attachments. Workers do this by default and no configuration is necessary. The connector can be configured to automatically sort emails after analysis into user-defined email folders in the configured Exchange user account. ### Configuring the Exchange user account To configure the connection with the Exchange user account: - make sure the connector is enabled - fill in the fields in the *Exchange setup* section of the Email AbuseBox Connector dialog. | | | | | ---------------- | --------- | ------------------------------------------------------------ | | Server domain | Mandatory | Enter the domain or IP address of the Exchange server. The value should be FQDN, hostname or IP. This should not include the protocol (e.g., http) | | Email folder | Mandatory | Enter the name of the email folder from which the email messages will be collected for analysis. This folder must belong to the same Exchange user account for which the credentials are configured in this section. The folder name is case-sensitive. | | Connection Type | Mandatory | Supports IMAP (Basic Authentication) and Exchange (OAuth 2) methods of authentication. Depending on the selection, the next section of the form will ask for different user credentials: Basic Authentication asks for a username and password, OAuth 2 asks for a Client ID, Client Secret and Tenant ID. | | Email address | Mandatory | Enter the primary email address of the configured Exchange user account. | | Access Type | Mandatory | Delegate is used in environments where there’s a one-to-one relationship between users. Impersonation is used in environments where a single account needs to access many accounts. | | Connect securely | Optional | If selected, the connector will not accept connections to Exchange servers with untrusted or expired certificates. | Note that the connector will operate and analyze emails even if these advanced options are disabled. They only control the post-analysis activities. ## Network File Share Connector The Network File Share connector allows connecting up to five shared network resources to the appliance. Once the network shares are connected and mounted to the appliance, it can automatically scan the network shares and submit files for analysis. After analyzing the files, the appliance can optionally sort the files into folders on the network share based on their classification status. The Network File Share connector supports SMB and NFS file sharing protocols. Currently, it is not possible to assign a custom name to each network share. The only way to distinguish between configured network shares is to look at their addresses. If there are 3 configured network shares, and the network share 2 is removed, the previous network share 3 will automatically "move up" in the list and become network share 2. ### Configuring Network Shares To add a new network share to the appliance, expand the *Shares* section in the Network File Share Connector dialog and fill in the relevant fields. | | | | | ------------ | ------------------ | ------------------------------------------------------------ | | Address | Mandatory | Enter the address of the shared network resource that will be mounted to the appliance. The address must include the protocol (SMB or NFS). Leading slashes are not required for NFS shares (example: *nfs:storage.example.lan*). The address can point to the entire network drive, or to a specific folder (example: *smb://storage.example.lan/samples/collection*). When the input folder and/or sorting folders are configured, their paths are treated as relative to the address configured here. **Note:** If the address contains special characters, it may not be possible to mount the share to the appliance. The comma character cannot be used in the address for SMB shares. Some combinations of ? and # will result in errors when attempting to mount both the SMB and the NFS shares. | | Username | Optional, SMB only | Enter the username for authenticating to the SMB network share (if required). Usernames and passwords for SMB authentication can only use a limited range of characters (ASCII-printable characters excluding the comma). | | Password | Optional, SMB only | Enter the password for authenticating to the SMB network share (if required). Usernames and passwords for SMB authentication can only use a limited range of characters (ASCII-printable characters excluding the comma). | | Input folder | Optional | Specify the path to the folder on the network share containing the files to be analyzed by the appliance. The folder must exist on the network share. The path specified here is relative to the root (address of the network file share). If the input folder is not configured, the root is treated as the input folder. | ### Folder Naming Restrictions Folder names used in the **Address**, **Input folder** and **Automatic File Sorting** folder fields (described below) can include: - Alphanumeric characters (`A–Z`, `a–z`, `0–9`) - Spaces - The following special characters: `_`, `-`, `.`, `(`, `)` To ensure maximum compatibility across network file systems, avoid using special characters beyond underscore and hyphen. The following characters are **not allowed**: `/`, `\`, `:`, `*`, `?`, `"`, `<`, `>`, `|`, and the null byte (`\0`). When specifying subfolders in the **Input folder** or within the **Address**, each folder name must conform to these rules. --- The service will continually scan the network shares for new files (approximately every 5 minutes). If any of the existing files on the network share has changed since the last scan, it will be treated as a new file and analyzed again. ## Microsoft Cloud Storage: Azure Data Lake The Azure Data Lake connector allows connecting up to 20 Azure Data Lake Gen2 containers to the appliance. When the containers are connected and mounted to the appliance, it can automatically scan them and submit files for analysis. The files can be placed into the root of each container, or into an optional folder in each of the containers. **Important: This connector is not compatible with containers that have the Blob Soft Delete feature enabled.** After analyzing the files, the appliance can optionally sort the files into folders on the container based on their classification status. Currently, it is not possible to assign a custom name to each data lake input. The only way to distinguish between configured containers is to look at their names. If there are three configured data lake inputs, and input 2 is removed, the previous input 3 will automatically "move up" in the list and become input 2. ### Configuring Azure Data Lake containers To add a new Azure Data Lake container: - make sure the connector is enabled - expand the *Azure Data Lake Inputs* section in the Azure data lake dialog and fill in the relevant fields. | | | | | -------------------- | --------- | ------------------------------------------------------------ | | Storage account name | Mandatory | The name of the storage account. | | Storage access key | Mandatory | The access key used for Shared Key Authentication. This value should end in `==` | | Container | Mandatory | Specify the name of an existing Azure Data Lake container which contains the samples to process. The value must start and end with a letter or number and must contain only letters, numbers, and the dash (-) character. Consecutive dashes are not permitted. All letters must be lowercase. The value must have between 3 and 63 characters. | | Folder | Optional | The input folder inside the specified container which contains the samples to process. All other samples will be ignored. | ## Microsoft Cloud Storage: OneDrive / SharePoint Online The Microsoft Cloud Storage connector allows connecting up to five OneDrive or SharePoint storages to the appliance. When the storages are connected and mounted to the appliance, it can automatically scan them and submit files for analysis. The files can be placed into the root of each storage, or into an optional subfolder, After analyzing the files, the appliance can optionally sort the files into folders based on their classification status. ### Configuring File Storage Inputs To add a new File Storage Input: - make sure the connector is enabled - expand the *File Storage Inputs* section in the Microsoft Cloud Storage dialog and fill in the relevant fields. | | | | --------------- | ------------------------------------------------------------ | | Host | Host of the OAuth2 authentication server. | | Client ID | Identifier value of the OAuth2 client. | | Client Secret | The value used by the service to authenticate to the authorization server. | | Scopes | When the user is signed in, these values dictate the type of permissions Microsoft Cloud Storage connector needs in order to function. Provide one or more OAuth2.0 scopes that should be requested during login. These values should be separated by comma. | | Auth URL | Authentication endpoint for OAuth2. | | Token URL | Link to access token for OAuth2 after the authorization. | | Resource | The server that hosts the needed resources. | | Source | A choice between **OneDrive** and **Online Sharepoint**. | | Drive ID | Identifier value for the drive when **OneDrive** source option is chosen. | | Sharepoint Site | A dropdown which appears when the **Online Sharepoint** source option is chosen. | | Folder | The input folder which contains samples to be processed, while all other samples will be ignored. | ## AWS S3 Connector The S3 connector allows connecting up to 20 S3 buckets to the appliance. When the buckets are connected and mounted to the appliance, it can automatically scan the buckets and submit files for analysis. The files can be placed into the root of each bucket, or into an optional folder in each of the buckets. After analyzing the files, the appliance can optionally sort the files into folders on the S3 bucket based on their classification status. Currently, it is not possible to assign a custom name to each S3 file storage input. The only way to distinguish between configured buckets is to look at their names. If there are 3 configured S3 file storage inputs, and input 2 is removed, the previous input 3 will automatically "move up" in the list and become input 2. To ensure that all files in AWS S3 buckets can be reprocessed, users can use the **Clear All Processed Files** button. This option resets the connector’s tracking of previously processed files. As a result, all files in the buckets will be treated as new and reprocessed when the connector is restarted. No files in the buckets are deleted. ### Configuring S3 Buckets To add a new S3 bucket: - make sure the connector is enabled - expand the *S3 File Storage Inputs* section in the S3 dialog and fill in the relevant fields. | | | | | --------------------------- | --------------------------------------------------- | ------------------------------------------------------------ | | AWS S3 access key ID | Mandatory | The access key ID for AWS S3 account authentication. In cases where the appliance is hosted by ReversingLabs and Role ARN is used, this value will be provided by ReversingLabs. | | AWS S3 secret access key | Mandatory | The secret access key for AWS S3 account authentication. In cases where the appliance is hosted by ReversingLabs and Role ARN is used, this value will be provided by ReversingLabs. | | AWS S3 region | Mandatory | Specify the correct AWS geographical region where the S3 bucket is located. This parameter is ignored for non-AWS setups. | | Enable Role ARN | Optional | Enables or disables authentication using an external AWS role. This allows the customers to use the connector without forwarding their access keys between services. The IAM role which will be used to obtain temporary tokens has to be created for the connector in the AWS console. These temporary tokens allow ingesting files from S3 buckets without using the customer secret access key. If enabled, it will expose more configuration options below. | | Role ARN | Mandatory and visible only if `Role ARN` is enabled | The role ARN created using the external role ID and an Amazon ID. In other words, the ARN which allows the appliance to obtain a temporary token, which then allows it to connect to S3 buckets without using the customer secret access key. | | External ID | Optional, visible only if `Role ARN` is enabled | The external ID of the role that will be assumed. Usually, it’s an ID provided by the entity which uses (but doesn’t own) an S3 bucket. The owner of that bucket takes the external ID and creates an ARN with it. In non-production or test environments, you can enter a placeholder value for the External ID if your use case doesn't require a real one. This is useful when you do not want to enforce the External ID requirement while testing configurations. However, it is strongly recommended to use a valid External ID in production environments to maintain security. | | Role session name | Mandatory and visible only if `Role ARN` is enabled | Name of the session visible in AWS logs. Can be any string. | | ARN token duration | Mandatory and visible only if `Role ARN` is enabled | How long before the authentication token expires and is refreshed. The minimum value is 900 seconds. | | AWS S3 bucket | Mandatory | Specify the name of an existing S3 bucket which contains the samples to process. The bucket name can be between 3 and 63 characters long, and can contain only lower-case characters, numbers, periods, and dashes. Each label in the bucket name must start with a lowercase letter or number. The bucket name cannot contain underscores, end with a dash, have consecutive periods, or use dashes adjacent to periods. The bucket name cannot be formatted as an IP address. | | Processing Priority | Mandatory | Assign a priority for processing files from an S3 bucket on a scale of 1 (highest) to 5 (lowest). Multiple buckets may share the same priority. The default value is 5. | | AWS S3 folder | Optional | The input folder inside the specified bucket which contains the samples to process. All other samples will be ignored. The folder name can be between 3 and 63 characters long, and can contain only lower-case characters, numbers, periods, and dashes. Each label in the folder name must start with a lowercase letter or number. The folder name cannot contain underscores, end with a dash, have consecutive periods, or use dashes adjacent to periods. The folder name cannot be formatted as an IP address. If the folder is not configured, the root of the bucket is treated as the input folder. | | S3 endpoint URL | Optional | Enter a custom S3 endpoint URL. Specifying the protocol is optional. Leave empty if using standard AWS S3. | | Server Side Encryption Type | Optional | Specify the server-side encryption method managed by AWS for your S3 bucket. You can choose either “AES256” to enable Server-Side Encryption with Amazon S3-Managed Keys (SSE-S3) or “aws:kms” to enable Server-Side Encryption with AWS Key Management Service (SSE-KMS). This setting should be left blank unless your bucket policy requires SSE headers to be sent to S3. It is mutually exclusive with the Customer Encryption settings, meaning you should not configure this option alongside Customer Encryption Algorithm or Customer Encryption Key. | | Customer Encryption Algorithm | Optional | Defines the encryption algorithm used when you provide your own encryption keys. The only valid value for this field is “AES256”. It must be used in conjunction with the Customer Encryption Key field and cannot be used simultaneously with the Server Side Encryption Type. This option is intended for users who prefer to manage their own encryption keys rather than relying on AWS-managed keys. | | Customer Encryption Key | Optional | Provide a customer-managed encryption key for encrypting and decrypting objects in your S3 bucket. The key must be a valid Base64-encoded AES256 key. It must be used together with the Customer Encryption Algorithm field and is mutually exclusive with the Server Side Encryption Type. | | Connect securely | Optional | If selected, the connector will not accept connections to S3 buckets with untrusted or expired certificates. This setting only applies when a custom S3 endpoint is used. | | Enable Selection Criteria Using Metadata | Optional | If selected, the connector will fetch and process samples that either match the specified metadata or have no associated metadata. This option can be used to target specific samples for further ingestion and processing via Spectra Analyze. | | Classification | Optional | Specify classifications to ensure that only samples matching the selected classification criteria, or samples that have no associated metadata are considered for processing. The available classifications are `Unknown`, `Goodware`, `Suspicious`, `Malicious`. | | Threat Name | Optional | Specify only the samples containing any of the specified threat names in their metadata for further processing. Multiple threat names can be specified using the enter or tab key. | ## SMTP Connector The SMTP connector allows analyzing incoming email traffic on the appliance to protect users from malicious content. When enabled, the connector service collects emails (with attachments) and uploads them to the appliance for analysis. Each email message is saved as one file. If email uploading fails for any reason, the connector automatically retries to upload it to the appliance. When the analysis is complete, each email message receives a classification status from the appliance. In this operating mode, the connector acts as an SMTP Relay. Therefore, the connector should not be used as a front-end service for accepting raw email traffic, but only as a system inside an already established secure processing pipeline for SMTP email. To allow the SMTP connector to inspect and collect email traffic, users must ensure that the SMTP traffic in their network is diverted to port 25/TCP prior to configuring the connector on the appliance. **Warning: Additional port configuration may be required on the appliance. Because it involves manually modifying configuration files, this action can cause the appliance to malfunction. Contact [ReversingLabs Support](mailto:support@reversinglabs.com) for instructions and guidance.** **Profiles** There are two profiles for this connector: *Default* and *Strict*. These two profiles correspond to different Postfix configuration files. Clicking the underlined **See How It Affects Postfix Config** text will display a pop-up modal with a detailed *Default* and *Strict* profile Postfix configuration. In the *Default* profile, you don’t enforce TLS traffic and you accept any SMTP client. This corresponds to the following Postfix configuration: ``` mynetworks = 0.0.0.0/0 [::]/0 smtpd_tls_security_level = may smtp_tls_security_level = may ``` In the *Strict* profile, you do enforce TLS and you can also specify trusted SMTP clients (highlighted line 1 in the example below; see [Postfix docs](https://www.postfix.org/postconf.5.html#mynetworks) for the specific syntax). The relevant portion of the configuration looks like this in *Strict* mode: ``` mynetworks = 0.0.0.0/0 [::]/0 smtpd_tls_security_level = encrypt smtp_tls_security_level = encrypt smtpd_tls_mandatory_ciphers = high smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 smtpd_tls_mandatory_exclude_ciphers = aNULL, MD5 ``` **Starting the connector** After the connector is enabled, click the **Start connector** button. This will initiate the connector service on the appliance. **Pausing and disabling the connector** While the connector service is active, the *Start connector* button changes into **Pause connector**. Clicking this button temporarily halts the connector service, which in turn stops receiving and analyzing new email traffic. The connector service records the last state and is able to resume scanning when **Start connector** is clicked again. If the connector service is active during a scheduled or manually executed Purge action, the system will automatically stop the service before performing the Purge action, and start it after the Purge action is complete. To disable the entire connector service on the appliance, click **Disable connector** at the bottom of the page. When the connector is disabled, it will not be possible to reconfigure, start, or pause it until the service is enabled again. ### Use Case: Accept Email from Any Email Client or Server To configure the service to accept email from any email client or server, follow these steps: 1. **Enable the SMTP connector** 2. **Contact [ReversingLabs Support](mailto:support@reversinglabs.com)** with: - Your intended email forwarding address range - A formal request to enable email forwarding 3. After receiving your request, ReversingLabs will: - Create an MX DNS record for your hosted service - Configure receiver restrictions to accept email only from your domain 4. Once ReversingLabs has added the MX record to the DNS for your hosted service, **create email forwarding rules** in your email system to automatically forward emails to the hosted service for processing. ## Citrix ShareFile Connector The Citrix ShareFile connector allows integration with ShareFile to scan and classify files stored on the platform. It supports advanced options for sorting files based on classification (`Goodware`, `Malware`, `Suspicious`, `Unknown`) into designated folders on ShareFile. Additionally, users can enable automatic deletion of source files post-analysis. ### Configuring Citrix ShareFile To add a new Citrix ShareFile input: - make sure the connector is enabled. - expand the *Citrix ShareFile Inputs* section in the Citrix ShareFile dialog and fill in the relevant fields. | Input | Description | |---------------|---------------------------------------------------------------------------------------------------------------------------------------------| | Hostname | The hostname used to access ShareFile servers. Usually `.sharefile.com`. | | Client ID | Identifier value of the OAuth2 client. | | Client Secret | The value used by the service to authenticate to the authorization server. | | Username | The username for authenticating to the ShareFile servers. | | Password | The password for authenticating to the ShareFile servers. | | Root Folder | Root folder used for scanning, needs to be defined as GUID/Item Id in format `foxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx`, and available on Citrix ShareFile. If left empty, the connector will scan all the shared folders assigned to the user account, including the account's home folder, if it has one. | ## ICAP Server **The ICAP Connector acts as an ICAP server** and processes files sent through ICAP-enabled systems. It applies rules based on classifications, file size, and processing time, ensuring files meet the specified criteria before being analyzed by Spectra Detect. ### Supported Topologies The ICAP Server Connector is designed to integrate seamlessly in any ICAP deployment model, including but not limited to: - Application Delivery Controllers (ADCs) - Forward Proxy - Ingress Controller - Intrusion Prevention System - Load Balancer - Managed File Transfer - Next-Generation Firewall - Protecting Enterprise Storage - Reverse Proxies - Secure Remote Access - SSL Inspection and Termination - Web Application Firewall (WAF) - Web Gateway - Web Traffic Security ### Configuration Guides - [Spectra Detect integration with Kiteworks](/Integrations/ICAP/kiteworks/). ### Request and Response Modes The ICAP Server Connector supports two workflows: request mode (REQMOD) and response mode (RESPMOD). Configure your ICAP client to use one mode depending on your requirements: #### Request Mode (REQMOD) - Processes outgoing client requests before they reach the destination server. - Example: Validating uploaded files to ensure they meet security requirements. #### Response Mode (RESPMOD) - Processes server responses before they are delivered to the client. - Example: Scanning downloaded files for malware or sanitizing content before delivery. ### ICAP Response Codes The table below lists the ICAP response codes and their meanings: | ICAP Response Code | Description | |--------------------|-------------| | **100 Continue** | Signals the client to continue sending data to the ICAP server after the ICAP Preview message. | | **200 OK** | The request was successfully processed. This status is returned whether the file is blocked or not. If the file was blocked, the `X-Blocked` header will be set to `True`. In RESPMOD, a blocked file will result in a 403 HTTP status code. | | **204 No modification needed** | Returned when the file was not blocked and the client sent an `Allow: 204` header in the request. | | **400 Bad request** | Indicates a failure during file submission, analysis processing, or delivering results back to the client. | | **404 Not found** | The ICAP service could not be found. | | **405 Method Not Allowed** | The requested method is not supported on the endpoint. | | **500 Internal Server Error** | A generic server-side error occurred. | ### Configuring ICAP To configure the ICAP Server connector, ensure that it is enabled and adjust the settings in the ICAP Server configuration section. | | | | ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Max File Size | Specify the maximum file size (in megabytes) that the ICAP Connector will process. Files exceeding this size will not be analyzed. Default: `0 (unlimited)`. | | Allow Classifications | Select which classifications to allow. Available options: `goodware`, `unknown`, `suspicious`, `malicious`. Files not matching the selected classifications will be blocked. | | Service Alias | The base service name is `spectraconnector` and cannot be changed. Here you may define any additional aliases your ICAP client can use to connect. Do not re-enter `spectraconnector` here. | | Timeout | Set the timeout period (in seconds) for processing requests. If a file is not processed within this time, the request will be terminated. Default: `300`. | | REQMOD Block Page URL | Enter the full URL your ICAP clients will fetch when a request is blocked. **To use the default block page, set it to `https://{RL_APPLIANCE_IP}/icap-block-page` and replace `{RL_APPLIANCE_IP}` with your appliance’s actual IP or hostname.** If you host a custom block page, enter its URL here and ensure it's accessible to ICAP clients. **Note:** In some deployments, `RL_APPLIANCE_IP` may refer to a virtual IP or a load-balancer IP. | | RESPMOD Block Page | Upload a custom file that will replace the blocked HTTP response content. This page will be served to the client instead of the original response. **Maximum file size:** 0.5 MB. | | Use TLS | Select the checkbox to use a secure connection (TLS v1.3) when communicating with the ICAP server. | | ICAP server listen port | Specify the network port on which the ICAP server listens for incoming requests. The default is `11344` when TLS is enabled, and `1344` when TLS is disabled. | | Scan Raw Data | Enable this to extract the raw HTTP message body and send it to the ReversingLabs analysis engine unmodified. | ### File Source Tag via ICAP ICAP requests support the `X-System-Tag` header to track file origin and provide end-to-end traceability. When this header is included in an ICAP request, its value is automatically propagated to the analysis report and displayed in the Detections Overview table on the Spectra Detect Manager dashboard. The `X-System-Tag` header value is stored in the `source_tag` field within the `custom_data` section of the analysis report. Example ICAP request header: ``` X-System-Tag: ExampleTag ``` This results in the following metadata in the analysis report: ```json { "custom_data": { "source_tag": "ExampleTag" } } ``` Use the `source_tag` field to identify the line of business, department, or organizational unit from which files originate. This enables SOC teams to filter and aggregate submissions by source, improving investigation workflows and reducing time to resolution. ## SFTP Connector The SFTP Connector allows the appliance to automatically retrieve files from a remote SFTP server for analysis. Users can configure authentication using either a password or a public key. Once started, the connector continuously scans the specified folder and submits new files for analysis. Classification results can be viewed on the appliance after processing. | | | |---------------------|---------------------------------------------------------------------------------------------------| | Host/Server Address | Hostname or IP address of the target SFTP server. | | Port | Port used for the SFTP connection. Defaults to `22`. | | Username | Account name used to authenticate with the SFTP server. | | Authentication Type | Method used to authenticated with the SFTP server. Supported methods: `Password` or `Public Key`. | | Password | Password associated with the specified username. | | Input Folder | Path to remote directory from which files will be retrieved. Example: `/incoming/data` | **Tip: Key pairs can also be created using APIs. Refer to the API documentation on the appliance, specifically the **Appliances > Generate key pair** and **Appliances > Download public key** endpoints.** ## Using Advanced Connector Options In addition to main connector options for every connector service, users can set advanced options. Advanced options for a connector refer to actions that the connector service can perform on files after the appliance finishes analyzing them. Specifically, the connector can be configured to automatically sort files into user-defined sorting folders on the connector user account. Files are sorted into folders based on the classification status they receive during analysis (malicious, suspicious, known, no threats found). For Azure Data Lake, Network File Share, and S3 Connectors, advanced options can be configured for every storage unit individually. This means that the sorting criteria, folder names, and folder paths can be different on each configured storage unit. ### IMAP - MS Exchange - AbuseBox Connector | | | | ------------------------------- | ------------------------------------------------------------ | | Enable automatic message filing | Selecting the checkbox will allow the connector to move analyzed emails and sort them into email folders in the configured Exchange email user account. This checkbox toggles the availability of other options in the Advanced Options section. | | Malware folder | Specify the name of the email folder into which the connector will store emails classified as "Malicious" (malware). This folder will be created if it doesn’t exist. This field is mandatory when *Enable automatic message filing* is selected. | | Unknown folder | Specify the name of the email folder into which the connector will store emails with no malicious content detected (classified as Known, or not classified at all = Unknown). This folder will be created if it doesn’t exist. This field is mandatory when *Enable automatic message filing* is selected. | | Allow suspicious | When selected, emails classified as "Suspicious" will be moved to the configured Unknown folder. If this checkbox is not selected, files classified as "Suspicious" will by default be sorted into the configured Malware folder. | ### Azure Data Lake, Microsoft Cloud Storage, S3 Connectors | | | | ------------------------------------------------ | ------------------------------------------------------------ | | Enable Same Hash Rescan | Selecting the checkbox will force the connector to rescan samples that share the same hash. **This checkbox can only be enabled for the S3 connector.** | | Delete source files | Selecting the checkbox will allow the connector to delete source files on the connector storage after they have been processed. | | Enable automatic file sorting | Selecting the checkbox will allow the connector to store analyzed files and sort them into folders based on their classification. Usually, the connector skips already uploaded files. If this option is enabled and some files have already been uploaded, they will be uploaded to the Worker again. | | Sort Malware detected by Microsoft Cloud Storage | If enabled, the samples which are identified as Malware by Microsoft Cloud Storage will be moved to the Malware folder. These samples are not processed by Spectra Detect. **This checkbox can only be enabled for Microsoft Cloud Storage connector.** | | Goodware folder | Specify the path to folder into which the connector will store files classified as "Known" (goodware). This field is mandatory when *Enable automatic file sorting* is selected. The path specified here is relative to the address of the connector storage unit. If the folder doesn’t already exist on the container, it will be automatically created after saving the configuration. | | Malware folder | Specify the path to folder into which the connector will store files classified as "Malicious" (malware). This field is mandatory when *Enable automatic file sorting* is selected. The path specified here is relative to the address of the connector storage unit. If the folder doesn’t already exist on the container, it will be automatically created after saving the configuration. | | Unknown folder | Specify the path to folder into which the connector will store files without classification ("Unknown" status). The path specified here is relative to the address of the connector storage unit. If the folder doesn’t already exist on the container, it will be automatically created after saving the configuration. Files stored in the Unknown folder are regularly rescanned.| | Suspicious folder | Specify the path to folder into which the connector will store files classified as "Suspicious". The path specified here is relative to the address of the connector storage unit. If the folder doesn’t already exist on the container, it will be automatically created after saving the configuration. | ### Network File Share Connector | | | | ----------------------------- | ------------------------------------------------------------ | | Delete source files | Selecting the checkbox will allow the connector to delete source files on the network share after they have been processed. | | Enable automatic file sorting | Selecting the checkbox will allow the connector to store analyzed files and sort them into folders on every configured S3 bucket based on their classification status. This checkbox toggles the availability of other options in the Advanced Options section. | |Rescan Unknowns|If enabled, the connector rescans samples previously classified as unknown in intervals defined by the Rescan Unknowns Interval value (in days). If Allow Unknown is enabled, unknown files will be stored in the Goodware folder, and will not be rescanned.| | Goodware folder | Specify the path to folder into which the connector will store files classified as "Known" (goodware). This field is mandatory when *Enable automatic file sorting* is selected. The path specified here is relative to the address of the network file share. If the folder doesn’t already exist on the network share, it will be automatically created after saving the configuration. | | Malware folder | Specify the path to folder into which the connector will store files classified as "Malicious" (malware). This field is mandatory when *Enable automatic file sorting* is selected. The path specified here is relative to the address of the network file share. If the folder doesn’t already exist on the network share, it will be automatically created after saving the configuration. | | Unknown folder | Specify the path to folder into which the connector will store files without classification ("Unknown" status). The path specified here is relative to the address of the network file share. If this field is left empty, unknown files will be stored either to the Goodware or to the Malware folder, depending on the "Allow unknown" setting. | | Known threshold | Files classified as "Known" (goodware) with the trust factor value higher than the one configured here will be stored into the configured Malware folder. "Known" files with the trust factor less than or equal to the value configured here will be stored into the configured Goodware folder. Supported values are 0 to 5. Default is 5 (saves all to the Goodware folder). This field is mandatory when *Enable automatic file sorting* is selected. | | Allow unknown | When selected, files with the "Unknown" classification status are stored into the configured Goodware folder. If this checkbox is not selected, files with the "Unknown" status are either stored into the Unknown folder (if the Unknown folder is configured), or to the Malware folder (if the Unknown folder is not configured). | | Allow suspicious | When selected, files classified as "Suspicious" will be stored into the configured Goodware folder. If this checkbox is not selected, files classified as "Suspicious" will be stored into the configured Malware folder. | ## Global Configuration In addition to every connector service having specific configuration settings, there is a **Global Configuration** section at the bottom of every connector page. These settings apply to all configured connectors. | | | | -------------------------------------------------------- | ------------------------------------------------------------ | | Save files that had encountered errors during processing | Original files that were not successfully uploaded will be saved to `/data/connectors/connector-[CONNECTOR_SHORTNAME]/error_files/` | | Max upload retries | Number of times the connector will attempt to upload the file to the processing appliance. Upon reaching the number of retries, it will be saved in the `error_files/` destination or be discarded | | Max upload timeout | Period (in seconds) between upload attempts of the sample being re-uploaded. | | Upload algorithm | The algorithm used for managing delays between attempting to reupload the samples. In **Exponential backoff**, the delay is defined by multiplying the *Max upload timeout* parameter by 2, until reaching the maximum value of 5 minutes. **Linear backoff** will always use the *Max upload timeout* value for the timeout period between reuploads. | | Max upload delay | In case the Worker cluster is under high load, this parameter is used to delay any new upload to the cluster. The delay parameter will be multiplied by the internal factor determined by the load on the appliance. If set to 0, the delay will be disabled. | | Database cleanup period | Specifies the number of days for which the data will be preserved. | | Database cleanup interval | Specifies the time (in seconds), in which the database cleanup will be performed. | | Max File Size (MB) | The maximum file size the connector will transfer to the appliance for analysis. Setting it to 0 disables the option. (Available for AWS S3 and Azure Data Lake Connectors) | ## Unique Usernames | Connector | Unique Username | | ---------------------------- | ------------------------- | | Email AbuseBox Connector | abusebox_connector | | Azure Data Lake Connector | azure-data-lake_connector | | Network File Share Connector | fileshare_connector | | Microsoft Cloud Storage | graph-api-connector | | S3 Connector | s3_connector | | SMTP Connector | smtp_connector | | SFTP Connector | sftp_connector | --- ## Spectra Detect Appliance Configuration — Central Configuration and Hub Groups Spectra Detect Manager allows users to modify configuration settings on Spectra Detect appliances directly from the Manager interface. The Central Configuration feature makes it easier to configure appliances remotely, and to ensure that the settings are consistent and correct across multiple appliances. The appliances must first be connected and authorized on the Manager instance. **Note: **Spectra Analyze** appliances can be managed using the Spectra Detect Manager APIs.** To start working with this feature, access the Central Configuration page by selecting **Central Configuration** in the upper right corner of the Manager interface. The Central Configuration page contains different configuration modules that users can enable. Different combinations of modules and their configuration values can be saved as **configuration groups** or **Hub groups**. For example, users can create a configuration group for Worker appliances that should be connected to [Spectra Intelligence](/SpectraIntelligence/), and a Hub group for Hub and Worker appliances that should be connected to T1000. In addition to options described below, appliance groups containing a Hub instance provide more configuration options (such as Connector services) if they are configured as a [Spectra Detect Hub group](../Admin/Redundancy.md#hub-groups) rather than a regular configuration group. When appliances are added to a group, all settings configured in the modules are applied to them, overwriting their previous settings. Generally, the Central Configuration workflow includes the following steps: 1. Create a configuration group or edit an existing one. 2. Select which appliances should be in the group. 3. Modify settings in configuration modules. 4. Save changes 5. Apply modified settings to all appliances in the group. ## Central Configuration Page The Central Configuration page contains the *Select configuration group* pull-down list at the top, allowing users to switch between existing groups. There are also buttons for creating a new group and deleting the currently selected group. If there are no configuration groups created on the Manager instance, the `default` group is displayed on the Central Configuration page. Users can manage appliances and modify settings in the default group, or create their own groups. All configuration modules supported by the Manager are listed in the sidebar on the left. Selecting a module opens its configuration dialog on the right side of the page. ![Central Configuration page with configuration group selection and modules sidebar](../images/central-configuration.png) If the selected group is a [Spectra Detect Hub group](../Admin/Redundancy.md#hub-groups), an additional section is present at the top of the page. The section indicates which Hub instance in the group is the primary, and which is the fallback node. Clicking **Details** in this section displays more information about both Hub instances, such as their router IDs and configured priority values. To see the list of appliances that can be added to the currently selected configuration group, select *Appliances* in the sidebar. Appliances that are already in the current group have a tick in the checkbox next to their name. Appliances that are in other configuration groups have an indicator next to the appliance name. Users can save and/or apply configurations to appliances in the group by clicking on the **Save** button. This opens a pop-up dialog with the options to **Save** or **Save and Apply** the configuration to all appliances in the group. To apply the configurations to specific appliances, select their checkboxes in the appliance list below, and click the **Apply** button at the top of the list. Note that applying configuration removes any configurations that aren’t defined in the current configuration group, preventing mismatches caused by values from previous groups or manual changes. ### Configuration status The configuration status of appliances can be one of the following: - Applied - Not Applied - Pending - Error - Out of Sync **Note: Older appliances (i.e. before Spectra Detect v5.2) will show different status messages.** ## Adding Appliances to a Configuration Group Appliances that can be added to the current configuration group are listed in the *Appliances* section. Select the checkbox next to the appliance(s) that should be added to the group, and click **Save**. This opens a dialog with the options to save the selected appliances to the group, and to optionally apply the current group configuration. An appliance cannot be in two configuration groups at the same time. If an appliance is already in another configuration group, the Manager displays a warning message after clicking **Save**. Confirming the change will move the appliance from the previous configuration group to the current one. When an appliance is successfully added to a configuration group, the group’s configuration has to be manually applied to it either by clicking the **Save** button and selecting the **Save and Apply** option, or by selecting its checkbox in the Apply Configuration list and clicking the **Apply** button. The appliance will restart and start using the new configuration. The configuration dialogs on the appliance will indicate that the settings are being managed by the Manager. Although the configuration values modified in the group will still be editable in the appliance’s configuration dialogs, any changes saved on the appliance will not be applied as long as the appliance is managed through Central Configuration. If an appliance is added to a group and the configuration is applied, but the appliance is offline or unreachable by the Manager at that time, its settings will be modified when it becomes reachable. **Warning: Moving a Worker between Hub groups can result in configuration mismatches if the Worker’s settings do not align with the target Hub group. Applying configuration changes to the affected Worker, either through the GUI or the API, is expected to resolve the mismatches.** ## Creating Configuration Groups To create a new configuration group, click **Add new group** at the top of the Central Configuration page. **Tip: It’s also possible to create a configuration group by clicking **Add new group** on the Appliance Management tab on the Dashboard page.** Group names are case-sensitive, so "example" and "Example" are treated as two different groups. Supported characters for group names are `a-z`, `A-Z`, `0-9`, and the underscore ( `_` ). If the group name contains an unsupported character, an error message is displayed. Likewise, a warning is displayed when trying to create a configuration group with a name that is already taken by another group. The dialog also requires selecting the group type. Two types are supported: 1. Configuration group (for Spectra Detect Worker appliances without a Hub), 2. Hub group (for [setting up a high-availability cluster](../Admin/Redundancy.md#hub-groups)). Select the first type ("Configuration Group") and click **Add** to confirm changes and create a new configuration group. The newly created group will not contain any appliances, and there won’t be any active configuration modules. **Important: Some configuration modules and options apply only to specific appliance types or versions. For example, the "Splunk" configuration module and its options apply only to the Worker. Read more in the [configuration modules](#configuration-modules) section.** To enable a configuration module, select it in the sidebar on the Central Configuration page and modify the options in it. The indicator in the sidebar warns about unsaved changes in the module. Unsaved changes are lost if the user navigates away from the Central Configuration page without clicking **Save** first. Configuration modules that are not enabled do not have any indicators in the sidebar. Those that are enabled and properly configured have a green indicator. If there are issues with the configuration of a module, the indicator changes to red. Save changes in the module by clicking **Save**. The indicator in the sidebar shows whether the module is configured properly. Repeat this procedure for every configuration module that needs to be enabled in the configuration group. To disable a configuration module, select it in the sidebar and click **Reset to Default**. This action removes all current configuration entries and restores the default settings in the UI. The changes are temporary until you **save** and then click **Apply**. Applying pushes the new settings to all connected appliances. The full list of supported configuration modules and options for all appliance types is available in the [configuration modules](#configuration-modules) section. ## Managing Configuration Groups The following changes can be made to any configuration group on the Manager: - enable and disable configuration modules - change options in enabled configuration modules - add and remove appliances from a group - move appliances from one group to another - delete the entire group (does not apply to the *default* group, which cannot be deleted) **Depending on the type of change, appliances may be automatically restarted. Only applying new configurations to an appliance will trigger a restart of that specific appliance. Adding an appliance to a group, removing it from a group, moving it between groups, or deleting a group will not restart the appliances.** Depending on the type of appliance, the process of restarting and reloading configuration values might take some time. Spectra Detect Worker appliances generally take longer to restart. ## Configuration modules The configuration modules listed in this section can be enabled on the Central Configuration page, and their options can be modified. Some configuration modules support all versions of Spectra Detect appliances, but specific options within the modules apply only to specific versions. Such options are indicated by a comment in the Manager interface. ### General Root SSH login can be enabled for use with password management systems. These checkboxes are not available by default. To enable them, do the following: 1. Log in via SSH to the Manager. 2. Run `sudo rlapp configure --sshd-control-enable`. This will enable the checkboxes on the Manager. 3. In the browser, go to *Spectra Detect Manager > Central configuration*, select the Hub group which will have root SSH login enabled, then go to *General > SSH* 4. Enable *Permit SSH configuration* 5. Enable *Permit root SSH login* Note that this can only be applied to Hub groups. For SSH credentials, contact [ReversingLabs Support](mailto:support@reversinglabs.com) . This section also includes the option to disable the use of swap memory. Swap memory is disk space used as RAM. Note that this option isn’t applicable if the appliances are deployed as Docker images. ### [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis) This section lists configuration settings related to static analysis performed by Spectra Core. #### Processing Settings This setting determines which file formats will be unpacked by Spectra Core for detailed analysis. "Best" fully processes all formats supported by the appliance. "Fast" processes a limited set of file formats. Fast option does not support unpacking and/or validation of several file formats, providing only minimal information for: - Archives (ActiveMimeMSO, ARC (.arc, .ark), ARSC, BLZ, CGBI, CRTF, DICOM (.dicom, .dcm, .dic), PE .Net Resources, LZ4, LZIP, LZMA, LZOP, MAR, NuGet, PAK, PCAP (http, smtp), PYZ, SQX, TIFF, WARC, XAR, ZOO) - Documents (bplist, Certutil (.crt, .cert, .pem), CHM, HTML (.html, .htm, .xhtml, .xht), IQY, SettingContent (.xml), SYLK, URL) - Mobile Applications (Android (.apk), iOS (.ipa), Windows Phone (.xap), APPX) - Multimedia (BMP, GIF, JPEG, PNG, SWF) - File System/Firmware (cramfs, HFSP, jffs2, squashfs, yaffs) - Web Applications (Google Chrome (.crx), Opera (.oex), Mozilla FireFox (.xpi)) - Quarantine formats (KasperskyKLQ, SymantecQBD, SymantecVBN) - Emails (UUE, YENC) - Disk Images (VHD, WIM (.wim, .swm)) - ...and others (CxMacro, Docker, PyInstaller, SMTP, sqlite3 (.db, .sqlite), VBE(.vbe, .jse)). Additionally, the report metadata will no longer include overlay and resources hashes, storyteller descriptions, Spectra Intelligence XREF data, Mitre ATT&CK mappings, IoC reasons, as well as mobile, browser and media details. #### CEF Messages Configuration Spectra Detect can log events using the Common Event Format (CEF) to ensure compatibility with security information and event management software products (SIEMs). CEF is an extensible, text-based logging and auditing format that uses a standard header and a variable extension, formatted as key-value pairs. Select the checkbox to enable sending CEF messages to a syslog receiver. **CEF Message Hash Type**: The hash type to use for CEF messages. #### String Extraction Configuration **Note: Changing the strings configuration can impact classification.** Spectra Core can extract information from binaries in the form of strings. While useful in some contexts, this metadata can also be very extensive. **Minimum String Length**: The minimum length of extracted strings that make it into the analysis report. Default is 4. **Maximum String Length**: The maximum length of extracted strings that make it into the analysis report. Default is 32768. A value of 0 is interpreted as unlimited length, and can increase processing memory requirements. Additionally, the following options can be enabled: Unicode Printable, UTF-8 Encoding, UTF-16LE Encoding, UTF-16BE Encoding, UTF-32LE Encoding, UTF-32BE Encoding. #### MWP-related settings **MWP goodware factor**: The value configured here determines the threshold at which the KNOWN classification for a file (from the Malware Presence algorithm) will change to the Spectra Core Goodware classification. By default, all KNOWN classifications are converted to Goodware. Lowering the value reduces the number of files classified as goodware. Files with a trust factor higher than the configured value are considered to have no threats. Supported values are 0 - 5. The default is 2. **Extended MWP Metadata**: Select the checkbox to include extended malware presence metadata in Worker analysis reports for files analyzed with AV engines in the Spectra Intelligence system. Spectra Detect Worker must be connected to Spectra Intelligence, and the user account must have appropriate access rights for this feature to work. Note that extended metadata is displayed in the report only for those files that have been analyzed by AV engines at some point. #### Decompression configuration Decimal value between 0 and 999.9. Used to protect the user from intentional or unintentional archive bombs, terminating decompression if size of unpacked content exceeds a set quota. The maximum allowed decompression ratio is calculated as: ``` MaximumDecompressionFactor * (1000 / ln(1 + InputFileSize * pow(10, -5))) ``` The `InputFileSize` must be in bytes. To calculate the maximum decompressed file size, multiply this ratio by the `InputFileSize`. Unpacking will stop once the size of all extracted content exceeds the theoretical maximum of the best-performing compression algorithm. Setting it to 0 will disable decompression management. ReversingLabs recommend against disabling decompression management. #### Propagation When propagation is enabled, files can be classified based on the content extracted from them. This means that files containing a malicious or suspicious file will also be considered malicious or suspicious. Goodware overrides ensure that any files extracted from a parent file and whitelisted by certificate, source or user override can no longer be classified as malicious or suspicious. Additionally, this goodware classification can be propagated from extracted files to their parent files in order to prevent and suppress possible false positives within highly trusted software packages. Goodware overrides will apply to all files with the trust factor value equal to or lower than the value configured here. Trust factor is expressed as a number from 0 to 5, with 0 representing the best trust factor (highest confidence that a file contains goodware). #### Enable Classification Scanners Fine-tune which scanners are used in the static analysis performed by Workers. - Images: heuristic image classifier - PECOFF: Heuristic Windows executable classifier - Documents: Document format threat detection - Certificates: Checks whether the file certificate passes the certificate validation in addition to checking white and black certificate lists - Hyperlinks: Embedded hyperlink threat detection - Emails: Phishing and email threat detection #### ML Model Configuration Configure the behavior of machine learning models used in static analysis. Each model can be set to one of the following options: - **Ignored**: The model's output is not considered in the classification decision. - **Disabled**: The model is not executed during analysis. - **Suspicious**: Detections from this model result in a suspicious classification. - **Malicious**: Detections from this model result in a malicious classification. The default option for all models is **Malicious**. **Important: General models are early warning detectors that can identify novel malware. Being both predictive and more aggressive than specialized models, they can result in unwanted detections for legitimate software interacting with low-level system functions.** ##### Scripts | Option | Description | Example Detection | | ------ | ----------- | ----------------- | | Python | ML model for detecting malicious Python scripts. | Script-Python.Malware.Heuristic | | Visual Basic | ML model for detecting malicious Visual Basic scripts. | Script-Macro.Malware.Heuristic | | PowerShell | ML model for detecting malicious PowerShell scripts. | Script-PowerShell.Malware.Heuristic | | AutoIT | ML model for detecting malicious AutoIT scripts. | Script-AutoIt.Malware.Heuristic | | Excel | ML model for detecting malicious Excel macros. | Script-Macro.Malware.Heuristic | ##### Windows | Option | Description | Example Detection | | ------ | ----------- | ----------------- | | General | General ML model for detecting malicious Windows executables. | Win[32\|64].Malware.Heuristic | | Backdoor | ML model for detecting backdoor threats in Windows executables. | Win[32\|64].Backdoor.Heuristic | | Downloader | ML model for detecting downloader threats in Windows executables. | Win[32\|64].Downloader.Heuristic | | InfoStealer | ML model for detecting information stealer threats in Windows executables. | Win[32\|64].Infostealer.Heuristic | | Keylogger | ML model for detecting keylogger threats in Windows executables. | Win[32\|64].Keylogger.Heuristic | | Ransomware | ML model for detecting ransomware threats in Windows executables. | Win[32\|64].Ransomware.Heuristic | | Riskware | ML model for detecting riskware threats in Windows executables. | Win[32\|64].PUA.Heuristic | | Worm | ML model for detecting worm threats in Windows executables. | Win[32\|64].Worm.Heuristic | ##### Linux | Option | Description | Example Detection | | ------ | ----------- | ----------------- | | General | General ML model for detecting malicious Linux executables. | Linux.Malware.Heuristic | #### Ignore the Following Threat Types Selected threat types will be excluded from final classification decision. The classification returned will be Goodware with reason Graylisting. - Adware - Packer - Riskware - Hacktool - Spyware - Spam #### Password List Appliances use these passwords to decrypt password-protected compressed files submitted for analysis. Prior to submitting password-protected compressed files to the appliance, users can add the password for each file to this list (one password per line). Passwords can also be provided on each upload using the optional `user_data` request field. ### Spectra Intelligence **Applies to Spectra Detect Worker** | Option | Description | | -------------------------------------------------- | ------------------------------------------------------------ | | Enable Spectra Intelligence | Receive additional classification from the Spectra Intelligence cloud. By default, it is false. | | Spectra Intelligence URL | The host address for the Spectra Intelligence cloud. Click the *Test connection* button to test the connectivity. The default URL is *https://appliance-api.reversinglabs.com* | | Username | Spectra Intelligence username | | Password | Spectra Intelligence password | | Timeout | Default Spectra Intelligence connection timeout in seconds (maximum 1000). | | Enable proxy | Enables the configuration of an optional proxy connection. By default, it is false. | | Proxy host | Proxy host name for routing requests from the appliance to Spectra Intelligence (e.g., 192.168.1.15). | | Proxy port | Proxy port number (e.g., 1080). | | Proxy username | User name for proxy authentication. | | Proxy password | Password for proxy authentication. | Cache Spectra Intelligence results to preserve quota and bandwidth when analyzing sets of samples containing a lot of duplicates or identical extracted files. | Option | Description | |---------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Enable | Enable or disable the caching feature. Default: False | | Cache max size (%) | Maximum cache size expressed as a percentage of the total allocated RAM on the Worker. Default: 6.25, Range: 5 - 15 | | Cache cleanup window | How often to run the cache cleanup process, in minutes. It is advisable for this value to be lower, or at least equal to the TTL value. Default: 10, Range: 5 - 60 | | Maximum number of idle upstream connections | The maximum number of idle upstream connections. Default: 50, Range: 10 - 50 | | Cache entry TTL | Time to live for cached records, in minutes. Default: 240, Range: 1 - 3600 | Deep Cloud Analysis enables uploading files to Spectra Intelligence for scanning, making the results available in real-time on the Spectra Detect Dashboard. You can enable or disable Deep Cloud Analysis in the **Administration > Spectra Detect Manager > Dashboard Configuration** section by clicking the "Enable Deep Cloud Analysis" checkbox. | Option | Description | |--------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Wait for Deep Cloud Analysis Results | Delays report generation until the latest AV scan completes, ensuring updated AV metadata is included. This option is disabled by default. | | Deep Cloud Analysis Timeout | Maximum time in minutes AV waits for a scan to complete before timing out, ensuring system performance isn't impacted by long delays. Default: 240. Minimum value: 1. Maximum value: 1440. | | Scan Unpacked Files | Sends unpacked files to Deep Cloud Analysis for scanning. | | Rescan Files on Submission | Rescans files upon submission based on the configured interval to include the latest AV results in the reports. This option is enabled by default. | | Rescan Interval | Set the interval in days for triggering an AV rescan. If the last scan is older than the specified value, a rescan will be initiated. A value of 0 means files will be rescanned with each submission. Default: 3. | | File type filter | Select which file types should be excluded from Deep Cloud Analysis. `PE` maps to `PE16`, `PE`, `PE+`, `ELF` maps to `ELF32 Little`, `ELF32 Big`, `ELF64 Little`, `ELF64 Big`, `MachO` maps to `MachO32 Little`, `MachO32 Big`, `MachO64 Little`, `MachO64 Big`, `Package/JAR` maps to `Package/Java/JAR` and `Binary/Archive/JAR`. | | File name filter | Define which file name patterns should be excluded from Deep Cloud Analysis. Use wildcard (*) to match multiple characters. | ### T1000 File Reputation Appliance **Applies to Spectra Detect Worker** | Option | Description | | -------------- | ------------------------------------------------------------ | | Enable T1000 | When enabled, an integration with ReversingLabs T1000 instance to receive additional classification information is configured. By default, it is false. | | T1000 URL | The host address for the on-premises T1000 File Reputation appliance. | | Username | T1000 user name for authentication. Note: this user name needs to be created via the T1000 Web administration application. | | Password | T1000 password for authentication. | | Timeout | Default T1000 service connection timeout in seconds (maximum 60). | | Enable proxy | Enables the configuration of an optional proxy connection. By default, it is false. | | Proxy host | Proxy host name for routing request from the appliance to T1000 (e.g., 192.168.1.15). | | Proxy port | Proxy port number (e.g., 1080). | | Proxy username | User name for proxy authentication. | | Proxy password | Password for proxy authentication. | ### SNMP **Applies to Spectra Detect Worker** | Option | Description | | -------------------- | ------------------------------------------------------------ | | Enable SNMP service | Select the checkbox to enable the Simple Network Management Protocol service. | | Community | Enter the name of an SNMP community list for authentication. Community is a list of SNMP clients authorized to make requests. The SNMP service will not function properly if this field is not configured. | | Enable trap sink | Select the checkbox to enable sending SNMP traps to the sink server. Traps are asynchronous, unsolicited SNMP messages sent by the SNMP agent to notify about important events on the appliances. | | Trap community | Enter the SNMP trap community string. If the *Enable SNMP service* and *Enable trap sink* checkboxes are selected, then this field is required. | | Trap sink server | Enter the host name or the IP address of the trap sink server. The sink server is the location to which SNMP traps will be sent. If the *Enable SNMP service* and *Enable trap sink* checkboxes are selected, then this field is required. | | SNMP trap thresholds | A set of configuration fields allowing the user to set the thresholds (values that will trigger an SNMP trap) for supported types of events. Thresholds can be configured for average system load in 1, 5, and 10 minutes (as a percentage), used memory and used disk space (as a percentage), the size of Spectra Detect queues (maximum value is 20000) and the size of the classifications queue (maximum value is 20000). | ### System Time **Applies to Spectra Detect Worker** | Option | Description | | ----------------------------------- | ------------------------------------------------------------ | | Enable network time synchronization | Select the checkbox to enable server clock synchronization via NTP, which uses port 123. | | NTP servers | A list of servers, one per line, to use for system clock synchronization. | ### Spectra Detect Worker Configuration #### General ##### Limits It is possible to set up limits on file processing: - maximum file size - number of daily uploads File size is in MB, and the daily limit includes files uploaded through a connector. It also resets at midnight. **Large Report Size Limit (MB)** - Reports over this size will be handled by optimizing memory consumption (RAM), which may result in longer processing and post-processing times. Views are not supported for the full report; they can only be used with the split report option. Use this option when minimizing memory usage is important. Setting to 0 disables this option. ##### Health Monitoring **Processing and Postprocessing Service Status Check** *Processing* and *postprocessing* service status fields can be used to configure how often the services will be checked for timeouts. If any issues are detected, the process will be restarted. The default for both fields is 720 minutes. Setting to 0 will disable this option. **Monit Memory Threshold** *Monit Memory Threshold* is the percentage of memory, between 50 and 100, that services can use. If memory usage reaches the number configured here, the system will restart services. **If this number is set to 100, the restart will be disabled.** **Health Thresholds** Set the health thresholds to true or false to enable or disable the thresholds functionality. - **Disk High Threshold**: Specify the highest allowed percentage of hard disk usage on the system. If it exceeds the configured value, the appliance will start rejecting traffic. - **Queue High Threshold**: Specify the maximum number of items allowed in the queue. If it exceeds the configured value, the appliance will start rejecting traffic. Default: 100. ##### Cleanup *All values are in minutes* - File age limit How long an unprocessed file is present on the appliance before being deleted. Processed files are deleted immediately after processing. Default: 1440. - Task age limit How long before the record of a completed processing task is deleted. Default: 90. - Unprocessed task limit How long before an incomplete processing task is cancelled. Default: 1440. ##### Spectra Analyze Configuration - **Spectra Analyze IP address or FQDN**: Specify the hostname or IP address of Spectra Analyze appliance associated with the Worker. This address will be referenced in Splunk reports to enable retrieving additional processing information. #### File Processing ##### Processing - **Processing Mode**: Choose the processing mode of the Worker instance to improve pressure balancing. Supported modes are *standard* and *advanced*. In advanced mode, files larger than the threshold specified below are processed individually. - **Large File Threshold**: If advanced mode is selected, files larger than the threshold specified here will be processed individually, one by one. If standard mode is enabled, this parameter is ignored. The threshold value is expressed in MB. Default is 1000. Limit is 5000. - **Unpacking Depth**: Select how "deep" a file is unpacked. For example, if a file contains other files, each of those containing other files etc., by default (when this value is set to zero), Workers will unpack everything until no more files can be unpacked. Setting this value to something else than zero specifies the depth of recursion, which can be useful for quicker (but shallower) analyses. - **Processing Timeout**: Specify how many seconds Worker should wait for a file to process before terminating the task. The default is 28800 seconds (8 hours). The minimum allowed value is 1. ##### Caching - **Enable caching**: When caching is enabled, the SHA1 of file contents is used to determine if there have been recent analysis reports for the same file, and if those reports can be reused instead of processing the file again. - **Cache Timeout**: If file processing caching is enabled, this parameter can be used to specify for how long the analysis reports should be preserved in the cache and reused before they expire (in seconds). Restarting the Worker or changing configuration will clean the cache. Setting the value to 0 will use the timeout of 600 seconds. ##### Scaling - Processing Specify how many copies of Spectra Core instances to run. Changing this setting from the default is not recommended. - Post-processing Specify how many report post-processing instances to run. These instances will then modify and save reports as specified by the user. Increasing this value can increase throughput for servers with extra available cores. Default: 1. - Preprocessing Unpacker Specify how many copies of Spectra Core is used to unpack samples for Deep Cloud Analysis. This setting only has effect if Deep Cloud Analysis is enabled with the Scan Unpacked Files capability. *Applies only to Spectra Detect Worker v5.4.1 and higher* - Load size Defines the maximum number of individual files that can simultaneously be processed by a single instance of Spectra Core. When one file is processed, another from the queue enters the processing state. Default is zero (0), which sets the maximum number of files to be processed to the number of CPU cores on the system. - Concurrency Count Defines the number of concurrent threads per Spectra Detect instance that should be used for processing. Default is zero (0), which sets the number of threads to the number of CPU cores on the system. Modifying this option may cause issues with the system. Consult with ReversingLabs Support before making any changes to the parameter. #### Analysis Report ##### Default Report Settings - **Strings**: Select the checkbox to enable extracting strings from files during Spectra Detect file analysis. - **Relationships**: When enabled, the relationships section of the report lists hashes of files that are found within a given file. - **Relationships for First Report Only**: If disabled, the reports for samples that contain children files will include the relationships of all their descendants. This can lead to a lot of redundant information in the report. If enabled, relationship metadata will be included only for the root parent file. - **Network Reputation Report**: If enabled, Spectra Detect Worker (4.1+) file analysis reports will contain a new section, `network_reputation` , with reputation information on any network resources found within the file. This feature is unavailable if **Spectra Core > Processing Settings** is set to `Fast`, as it relies on interesting strings extracted during analysis. ##### API Report Settings This section configures the default report view applied to a Spectra Detect report if no other view has been applied elsewhere. Specify the report type that should be applied to the Worker analysis report. Report types are results of filtering the full report. In other words, fields can be included or excluded as required. - **Report Type**: Available report types are *extended_small*, *small*, *medium*, and *large*, as well as *classification*, *classification_tags*, *extended*, *mobile_detections* and *short_cert* which contain metadata equivalent to views with the same name. Click the *Upload* button to submit a custom report type to the appliance. - **Report View**: Apply a view for transforming report data to the *large* report type to ensure maximum compatibility. See **Spectra Detect Product Documentation > Analysis and Classification > Customizing Analysis Reports** for detailed information about how report types and views work. Enable the **Top Container Only** option to only include metadata for the top container. Reports for unpacked files will not be generated. Enable the **Malicious Only** option for the report to contain only malicious and suspicious children. - **Timeout (Seconds)**: Specify the timeout for API reports in seconds. The limit is 432000. Applies only to synchronous API calls. - **Number of Concurrent Connections**: Specify the number of concurrent connections for API reports. The limit is 5000. Applies only to synchronous API calls. ##### Additional hashes - CRC32 - MD5 - SHA384 - SHA512 - SSDEEP Spectra Core calculates file hashes during analysis and includes them in the analysis report. Select which additional hash types should be calculated for files analyzed on connected Worker appliances. MD5 is selected by default. SHA1 and SHA256 hashes are always included, and therefore aren’t configurable. Note that selecting additional hash types may cause the report to generate slower. #### Authentication ##### Tokens Specify tokens required for authorizing to the listed Spectra Detect Worker endpoints. Every token must be a string of alphanumeric characters between 16 and 128 characters in length. ### Egress Integrations After analysis, Spectra Detect can save: - original files - unpacked files - file analysis reports These are forwarded to one or more external storage providers: - AWS S3 - Network file share (SMB, NFS) - Microsoft Cloud storage (Azure Data Lake, OneDrive, SharePoint) #### Spectra Analyze Integration *Applies only to Spectra Detect Worker v5.4 and higher* Spectra Analyze integration allows Spectra Detect to upload processed samples to a configured Spectra Analyze instance. This can be used for sharing samples between Spectra Detect and Spectra Analyze, and for further analysis within a configured Spectra Analyze instance. - Enable Spectra Analyze Integration A checkbox to enable the Spectra Analyze integration. - Spectra Analyze Instance An instance of Spectra Analyze to which the Worker will upload samples. - Enable Global Filter A checkbox to enable the global filter set in the [Filter Management](../Admin/FilterManagement.md) section. When enabled, the Worker will only upload samples that pass the global filter criteria to Spectra Analyze. - Filter Name A filter that will be used to determine which samples are uploaded to Spectra Analyze. Samples uploaded to Spectra Analyze will have tags and comments visible on the Spectra Analyze Sample Summary page. - Supported tags: `filter_name`, `source_address`, `connector_name`, and `hostname`. - Supported comments: `full_file_path`. **Note:** If the upload is performed via the API, `source_address`, `connector_name`, and `full_file_path` will not be shown on the Spectra Analyze Sample Summary page. Clicking the `Create` button opens a filter creation dialog. The filter creation dialog follows the [Filter Management](../Admin/FilterManagement.md) workflow. #### AWS S3 There are two ways to connect to an output bucket: 1. Using your own S3 credentials. 2. Using an **IAM role**. The **AWS S3 Access Key ID** and **AWS S3 Secret Access Key** (in the *General* tab) must be provided in **both cases**. If ReversingLabs hosts the appliance and you use an IAM role, we will provide the access key and secret key. **Bucket naming conventions**: regardless of the type of storage (files, unpacked files, reports), input fields for S3 buckets expect the bucket names to conform to specific rules. The bucket name can be between 3 and 63 characters long, and can contain only lowercase characters, numbers, periods, and dashes. Each label in the bucket name must start with a lowercase letter or number. The bucket name cannot contain underscores, end with a dash, have consecutive periods, or use dashes adjacent to periods. The bucket name cannot be formatted as an IP address. ##### General - AWS S3 Access Key ID Specify your access key ID. - AWS S3 Secret Access Key Specify your secret access key. - AWS S3 Endpoint URL Enter S3 endpoint URL if you want to use S3 over HTTP. Only required in non-AWS setups in order to store files to an S3-compatible server. When this parameter is left blank, the default value is used ([https://aws.amazonaws.com](https://aws.amazonaws.com/)). Supported pattern(s): https?://.+ - SSL Verify This checkbox enables SSL verification in case of an `https` connection. - CA Path Path to the certificate file for SSL verification. If this parameter is left blank or not configured, SSL verification is disabled. By default, it is set to **/etc/pki/tls/certs/ca-bundle.crt**. - AWS S3 Region The default value is `us-east-1`. - AWS S3 Signature Used to authenticate requests to the S3 service. In most AWS regions, only Signature Version 4 is supported. For AWS regions other than `us-east-1` , the value `s3v4` must be configured here. (*Deprecated*) - AWS S3 Number of Upload Retries Maximum number of retries when saving a report to an S3-compatible server. ##### AWS IAM Role Settings Using S3 storage in a way where the customer secret key isn’t shared with the Spectra Detect system requires setting up an IAM role for Spectra Detect in the AWS console. This requires setting up the Amazon Resource Name (ARN) for Workers, which they can use to obtain temporary tokens. These temporary tokens allow saving files to S3 buckets without the customer secret access key. For this setup, an **external ID** is also required. This is provided by the entity which owns an S3 bucket. The owner of that bucket takes the AWS Account ID of the account that owns the appliance and builds an ARN with it (in the hosted Spectra Detect deployment, we provide our AWS Account ID). - ARN Session Name Name of the session visible in AWS logs. - Token duration How long the authentication token lasts before it expires and is refreshed. The minimum value is 900 seconds. - Refresh Buffer Number of seconds defined to fetch a new ARN token before the token timeout is reached. This must be a positive number, and the default value is 5. ##### File Storage This section configures how **original** files are stored. - Enable S3 File Storage A checkbox to enable storing files in an S3 bucket. - Store metadata This option stores the analysis metadata to the uploaded S3 object as key-value pairs. Metadata in S3 is used to attach additional information to objects. It can also be used to forward the data to another connector, appliance group, or Spectra Analyze for further processing. For example, you can use this metadata as a filter for only retrieving malicious files. By default, this option is enabled. For more details, see [AWS S3 Using Metadata](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingMetadata.html). - Enable Global Filter A checkbox to enable the global filter set in the [Filter Management](../Admin/FilterManagement.md) section. When enabled, the Worker will only upload samples that pass the global filter criteria. - Filter Name A filter that will be used to determine which samples are uploaded. Clicking the `Create` button opens a filter creation dialog. The filter creation dialog follows the [Filter Managemenet](../Admin/FilterManagement.md) workflow. - Output Bucket Destination There are two "modes" for setting the output bucket destination: 1. **Only Default Bucket**: storing files in the default bucket by inputting the bucket name. 2. **Map to Input Bucket**: storing files in buckets that are mapped to specific input buckets. - **Input Bucket**: A name of the input bucket that will be mapped to the output bucket. - **Output Bucket**: A name of the output bucket where the samples will be stored. If the output bucket is empty, any sample uploaded from the specified input bucket will be ignored. - **Mapping Filter name**: A filter that will be used to determine which samples are uploaded. Clicking the `Create` button opens a filter creation dialog. The filter creation dialog follows the [Filter Managemenet](../Admin/FilterManagement.md) workflow. After configuring the output to input bucket mappings, a table will be displayed with the columns *Input Bucket*, *Output Bucket*, *Filter*, and *Actions* (edit or delete), where users can view and manage the mappings. - Connection Method for Output Buckets Used to set individual AWS connection methods for target buckets. Clicking the `Add New Mapping` button opens a dialog where users can set up a new mapping. Multiple mappings can be set up. The dialog contains the following fields: - Buckets A list of input buckets that will be mapped to the output bucket. - Connection Strategy - **Standard AWS**: requires setting the *AWS S3 Access Key ID* and *AWS S3 Secret Access Key*, with additional options for setting the *AWS S3 Endpoint URL*, *AWS S3 Region*, *AWS S3 Signature* (deprecated in Spectra Detect version 5.4.0+), *AWS S3 Number of Upload Retries*, *Spectra Detect Worker AWS S3 Server-Side Encryption Algorithm*, verifying the HTTPS connection against the CA bundle, and setting the *CA Path*. - **ARN Connection**: requires selecting a strategy for role assumption: *Default AWS* or *Custom AWS Connection to Assume ARN Role*. Based on the selected role assumption strategy, input the configurations. - **Default AWS**: *Role ARN*, *External ID*, *ARN Session Name*, *Token Duration*, *Refresh Buffer*, *Spectra Detect Worker AWS S3 Servier-Side Encryption Algorithm*, *verify the HTTPS connection against the CA bundle*, and the *CA Path*. - **Custom AWS Connection to Assume ARN Role**: *AWS S3 Access Key ID*, *AWS S3 Secret Access Key*, *AWS S3 Endpoint URL*, *AWS S3 Region*, *AWS S3 Signature* (deprecated in Spectra Detect version 5.4.0+), *AWS S3 Number of Upload Retries*, *Role ARN*, *External ID*, *ARN Session Name*, *Token Duration*, *Refresh Buffer*, *Spectra Detect Worker AWS S3 Server-Side Encryption Algorithm*, *verify the HTTPS connection against the CA bundle*, and the *CA Path*. After configuring the output to input bucket mappings, a table will be displayed with the columns *Input Bucket*, *Output Bucket*, *Filter*, and *Actions* (edit or delete), where users can view and manage the mappings. Clicking the `Delete` button removes the selected mapping. - Server-Side Encryption Algorithm The server-side encryption algorithm can be any server-side encryption configured on the target default bucket (such as `aws:kms` or `AES256`). Clicking the `Test Connection` button will attempt to verify the selected server-side encryption. - Folder Folder where samples will be stored on given S3 bucket. The folder can be up to 1024 bytes long when encoded in UTF-8. It must not start or end with a slash or contain leading or trailing spaces. Consecutive slashes ("//") are not allowed. The folder key containing relative path elements ("../") are valid if, when parsed left-to-right, the cumulative count of relative path segments never exceeds the number of non-relative path elements encountered. **Note: Storage age policy** Spectra Detect retains data stored in customer S3 buckets for **12 months**. After this period, the data is automatically deleted from the S3 storage. ##### Unpacked Files Storage This section configures how **unpacked** files are stored. There is just one **output mode**: a default bucket that needs to be specified in the *Bucket Name* field (with an optional *Folder* name). Unpacked files are saved in **two possible formats**: 1. Individual unpacked files are saved in subfolders, which can be named in three different ways: | Method | Format | | ------------------- | ---------------------------------------------- | | Date-based | `YYYY/mm/dd/HH` | | Date-and-time-based | `YYYY/mm/dd/HH/MM/SS` | | SHA-1-based | first four characters of a sample’s SHA-1 hash | 2. Unpacked files are saved as ZIP archives, which can optionally be password-protected. ##### Report Storage This section configures how **analysis reports** are stored. - Enable S3 Report Storage A checkbox to enable storing reports in an S3 bucket. - Enable Global Filter A checkbox to enable the global filter set in the [Filter Management](../Admin/FilterManagement.md) section. When enabled, mapping filters will be automatically disabled. - Filter Name A filter that will be used to determine which samples are uploaded. Clicking the `Create` button opens a filter creation dialog. The filter creation dialog follows the [Filter Managemenet](../Admin/FilterManagement.md) workflow. - Output Bucket Destination There are three "modes" for setting the output bucket destination: 1. **Map to Input Bucket**: storing reports in buckets that are mapped to specific input buckets. - **Input Bucket**: A name of the input bucket that will be mapped to the output bucket. - **Output Bucket**: A name of the output bucket where the samples will be stored. If the output bucket is empty, any sample uploaded from the specified input bucket will be ignored. - **Mapping Filter name**: A filter that will be used to determine which samples are uploaded. Clicking the `Create` button opens a filter creation dialog. The filter creation dialog follows the [Filter Managemenet](../Admin/FilterManagement.md) workflow. After configuring the output to input bucket mappings, a table will be displayed with the columns *Input Bucket*, *Output Bucket*, *Filter*, and *Actions* (edit or delete), where users can view and manage the mappings. 2. **Same as Input Bucket**: storing reports in the same bucket as the input bucket. 3. **Only Default Bucket**: storing reports in the default bucket by inputting the bucket name. - Connection Method for Output Buckets Used to set individual AWS connection methods for target buckets. Clicking the `Add New Mapping` button opens a dialog where users can set up a new mapping. Multiple mappings can be set up. The dialog contains the following fields: - Buckets A list of input buckets that will be mapped to the output bucket. - Connection Strategy - **Standard AWS**: requires setting the *AWS S3 Access Key ID* and *AWS S3 Secret Access Key*, with additional options for setting the *AWS S3 Endpoint URL*, *AWS S3 Region*, *AWS S3 Signature* (deprecated in Spectra Detect version 5.4.0+), *AWS S3 Number of Upload Retries*, *Spectra Detect Worker AWS S3 Server-Side Encryption Algorithm*, verifying the HTTPS connection against the CA bundle, and setting the *CA Path*. - **ARN Connection**: requires selecting a strategy for role assumption: *Default AWS* or *Custom AWS Connection to Assume ARN Role*. Based on the selected role assumption strategy, input the configurations. - **Default AWS**: *Role ARN*, *External ID*, *ARN Session Name*, *Token Duration*, *Refresh Buffer*, *Spectra Detect Worker AWS S3 Servier-Side Encryption Algorithm*, *verify the HTTPS connection against the CA bundle*, and the *CA Path*. - **Custom AWS Connection to Assume ARN Role**: requires setting the *AWS S3 Access Key ID*, *AWS S3 Secret Access Key*, *AWS S3 Endpoint URL*, *AWS S3 Region*, *AWS S3 Signature* (deprecated in Spectra Detect version 5.4.0+), *AWS S3 Number of Upload Retries*, *Role ARN*, *External ID*, *ARN Session Name*, *Token Duration*, *Refresh Buffer*, *Spectra Detect Worker AWS S3 Server-Side Encryption Algorithm*, verifying the HTTPS connection against the CA bundle, and setting the *CA Path*. Optionally, you can configure the *SNS Topic ARN* to send SNS notifications to this topic when files are uploaded to these buckets (only applies to Worker >= v6.0.1). After configuring the output to input bucket mappings, a table will be displayed with the columns *Input Bucket*, *Output Bucket*, *Filter*, and *Actions* (edit or delete), where users can view and manage the mappings. Clicking the `Delete` button removes the selected mapping. - Folder Folder where reports will be stored on given S3 bucket. The folder can be up to 1024 bytes long when encoded in UTF-8. It must not start or end with a slash or contain leading or trailing spaces. Consecutive slashes ("//") are not allowed. The folder key containing relative path elements ("../") are valid if, when parsed left-to-right, the cumulative count of relative path segments never exceeds the number of non-relative path elements encountered. - Report Type / View Upload, manage and configure report types. See **Spectra Detect Product Documentation > Analysis and Classification > Customizing Analysis Reports** for detailed information about how report types and views work. - Top Container Only Enable to only include metadata for the top container. Reports for unpacked files will not be generated. - Malicious Only For the report to contain only malicious and suspicious children. - Split Report Split reports of extracted files into individual reports. - Archive and Compress Split Report Files Enable sending a single, smaller archive of split report files to S3 instead of each file. Relevant only when the "Split report" option is used. - Archive Password If set, enables encryption of the archive file using this value as the password. Relevant only when the "Archive and compress split report files" option is used. - Subfolder Reports can be saved into subfolders, with specific naming formats: | Subfolder naming | Format | |---------------------|------------------------------------------------| | Date-based | `YYYY/mm/dd/HH` | | Date-and-time-based | `YYYY/mm/dd/HH/MM/SS` | | SHA-1-based | first four characters of a sample’s SHA-1 hash | - Enable Filename Timestamp This refers to the naming of the report file itself (and not the subfolder in which it is stored). A timestamp is appended to the SHA1 hash of the file. The timestamp format must follow [the strftime specification](https://strftime.org/) and be enclosed in quotation marks. If not specified, the ISO 8601 format is used. ##### SNS You can enable publishing notifications about file processing status and links to Worker analysis reports to an Amazon SNS (Simple Notification Service) topic. The configured AWS account must be given permission to publish to this topic. The topic name is limited to 1000 characters. #### Network Share Configuration Specify the protocol and the address of the network share. The supported protocols are NFS and SMB, so for an NFS share, the address would start with `nfs://`. The address should include the IP address or URL of the share, and should also include the full path to the shared directory. Note: the *username* and *password* fields are not required for NFS, only for SMB. #### Microsoft Cloud Storage Configuration ##### Azure To set up saving on Azure, specify the storage account name and storage account key. In order to use a custom server, specify the endpoint suffix for the address of your Azure Data Lake container, which defaults to **core.windows.net**. The Azure integration can save analyzed files, unpacked files, as well as file analysis reports. The integration **doesn’t** support saving unpacked files or split reports into a single ZIP archive. Regardless of the use, the container name must be a valid DNS name, conforming to the following naming rules: 1. Container names must start or end with a letter or number, and can contain only letters, numbers, and the dash (-) character. 2. Every dash (-) character must be immediately preceded and followed by a letter or number; consecutive dashes are not permitted in container names. 3. All letters in a container name must be lowercase. 4. Container names must be from 3 through 63 characters long. ##### OneDrive / SharePoint Online Microsoft Cloud Storage allows saving of reports and analyzed files on either SharePoint Online or OneDrive storage types. After configuring the *Client ID*, *Client Secret*, and the application’s *Custom Domain* (as configured in the Azure portal), select the desired storage type and fill in the appropriate additional credentials: OneDrive storage requires the OneDrive account *Username*, while SharePoint requires the *Site Hostname* and *Site Relative Path* (for example, `sharepoint.com` and `sites/test-public`). #### Saving Files and Reports Different integrations have different options available for saving analyzed files and reports, but all will require configuring the storage location (S3 bucket or buckets, Microsoft Cloud storage, Azure container, or network share) and a folder on that storage (optional). These settings relate to where Spectra Detect Workers appliances save files. ##### Report customization It’s possible to change the look of the report to further customize it. See **Spectra Detect Product Documentation > Analysis and Classification > Customizing Analysis Reports** for detailed information about how report types and views work. ##### Splitting reports All storage options allow splitting the report into several JSON files, one report per extracted file. In addition to that, S3 and network share sections also allow saving these individual reports together in an archive (optionally password-protected). ##### Report name format Reports are saved with their timestamp at the end of their file name. By default, they will end with an ISO 8601 datetime string (YYYY-MM-DDTHH:mm:ss.SSSSSS) but this can be modified following the Python `strftime()` syntax. For example, to save reports only with their year and month, set the *Filename Timestamp Format* to `%Y%m`. This field is editable only if the **Enable Filename Timestamp** option is turned on. ### Saving unpacked files During analysis, a Worker can extract "children" from a parent file (the file initially submitted for analysis). Such child files can be saved to one of the external storage options (S3 bucket, Azure container, or network share). It’s also possible to sort them into subfolders based on date, date and time, or based on the first four characters of the sample’s SHA-1 hash. S3 and network share sections also allow saving these unpacked files as a ZIP archive instead of each individual file (*Archive and Compress Unpacked Files*). #### Callback Select the checkbox to enable sending file analysis results to an HTTP server ("callback"), and optionally return the analysis report in the response. Specify the full URL that will be used to send the callback POST request. Only HTTP is supported. If this parameter is left blank, no requests will not be sent. Additionally, specify the number of seconds to wait before the POST request times out. Default is 5 seconds. In case of failure, the Worker will retry the request up to six times, increasing waiting time between requests after the second retry has failed. With the default timeout set, the total possible waiting time before a request finally fails is 159 seconds. ##### CA path If the Callback URL parameter is configured to use HTTPS, this field can be used to set the path to the certificate file. This automatically enables SSL verification. If this parameter is left blank or not configured, SSL verification will be disabled, and the certificate will not be validated. ##### Report options and views It’s possible to change the look of the report to further customize it. Click the *Upload* button to submit a custom report type, or select one of the default options. Names for views can only include alphanumerical characters, underscores and dashes. See **Spectra Detect Product Documentation > Analysis and Classification > Customizing Analysis Reports** for detailed information about how report types and views work. Enable the **Top Container Only** option to only include metadata for the top container. Reports for unpacked files will not be generated. Enable the **Malicious Only** option for the report to contain only malicious and suspicious children. Enable the **Split Report** option to split reports of extracted files into individual reports. Enable the **Include Full Report** option to retrieve the full report. By default, only the summary report is provided in the callback response. ### Archiving *only for S3 and Azure* Files can be stored either as-is or in a ZIP archive. This archive can further be password-protected and customized: - Zip Compression Level: 0 (no compression) to 9 (maximum compression). The default is 0. - Maximum Number of Files: Specify the maximum allowed number of files that can be stored in one ZIP archive. Allowed range: 1 … 65535 ### File filters **Tip: You can also configure [advanced file filters](../Admin/FilterManagement.md).** The file filter is used by the Worker to control which files won't be stored after processing. You can filter out files based on their classification, factor, file type, and file size. For a file to be filtered out by the Worker, at least one of the filters has to match. To enable the feature, select the *File filters* checkbox in the **Central Configuration ‣ Egress Integrations ‣ File Filters** dialog. Then, use the **Add new filter** button to create custom file filters. Every filter can be individually enabled or disabled by selecting or deselecting the *Active* checkbox, or the checkbox to the right of the filter name. All created filters are listed in the dialog. Every filter can be expanded by clicking the arrow to the left of the filter name. When a filter is expanded, users can modify any of the filtering criteria, or remove the entire filter by clicking **Delete**. #### File Filtering Criteria | CRITERIA | DESCRIPTION | | -------------- | ------------------------------------------------------------ | | Classification | Allows the user to filter out files by their classification. Supported values are "Known" and "Unknown". Both values can be provided at the same time. Malicious and suspicious files cannot be filtered out. | | Factor | Allows file filtering based on threat factor. When a file is processed, it is assigned a threat factor value, represented as a number from 0 to 5, with 5 indicating the most dangerous threats (highest severity). Enter one value from 0 to 5. The filter program will filter out files with the threat factor of N (entered value) or less. | | File Type | Spectra Detect Worker can identify the file type for every analyzed file. To filter out files by type, select one or more file types, or select the "All" checkbox. | | File Size | To filter out files by size, specify the file size in any of the supported units, and the file size condition (greater than or less than). The file size value should be provided as an integer; if it is not, it will automatically be rounded down to the nearest whole integer. | ### Splunk **Applies only to Spectra Detect Worker** | Option | Description | | ------------------------- | ------------------------------------------------------------ | | Enable | Select the checkbox to enable Splunk integration. | | Host | Specify the hostname or IP address of the Splunk server that should connect to the Worker appliance. | | Port | Specify the TCP port of the Splunk server’s HTTP Event Collector. Allowed range(s): 1 … 65535 | | Token | Specify the API token for authenticating to the Splunk server. Not mandatory. | | Use HTTPS | Select the checkbox to use the HTTPS protocol when sending information to Splunk. If it’s not selected, non-encrypted HTTP will be used. | | SSL require certificate | If HTTPS is enabled, selecting this checkbox will enable certificate verification. The Worker host needs to have correct certificates installed in order to successfully pass verification. | | Timeout - **Worker only** | Specify how many seconds to wait for a response from the Splunk server before the request is considered failed. If the request fails, the report will not be uploaded to the Splunk server, and an error will be logged. Default is 5 seconds | | Report type | Specify the report type that should be applied to the Worker analysis report before sending it to Splunk. Report types are results of filtering the full report. In other words, fields can be included or excluded as required. Click the *Upload* button to submit a custom report type to the appliance. Report types are stored in the `/etc/ts-report/report-types` directory on each Worker. | | Report view | Specify the name of an existing transformation view that should be applied to the Worker analysis report before sending it to Splunk. Views can be used to control the presentation format and the contents of the analysis report; for example, to flatten the JSON hierarchy, or to preserve only selected parts of the report. Allowed characters are alphanumeric characters, underscore and dash. Views are stored in the `/usr/libexec/ts-report-views.d` directory on each Worker. | | Top Container Only | Whether or not the report sent to Splunk should contain the reports for child files. | ### System Alerting **Applies to Spectra Detect Worker** | Option | Description | | ----------------------------------------- | ------------------------------------------------------------ | | **Syslog receiver** | | | Enable | Select the checkbox to receive alerts about the status of critical system services to a syslog server. Read more about which services are supported in the table below. | | Host | Host address of the remote syslog server to send alerts to. | | Port | Port of the remote syslog server. | | Protocol | Communication protocol to use when sending alerts to remote syslog server. Options are TCP (default) and UDP. | **System Alerting: Supported Services** Syslog notifications are sent when any of the services or operations meets the condition(s) defined in the table. | SYSTEM OPERATION OR SERVICE | NOTIFICATION TRIGGER | | ------------------------------- | ------------------------------------------- | | RAM | usage is over 90% for 10 minutes | | CPU | usage is over 40% for 2 minutes | | CPU wait (waiting for IO) | over 20% for 2 minutes | | Disk usage | over 90% for 10 minutes | | UWSGI service | down for 2 minutes | | NGINX service | down for 2 minutes | | RABBIT-MQ service | down for 2 minutes | | POSTGRES service | down for 2 minutes | | MEMCACHED service | down for 2 minutes | | CROND service | down for 2 minutes | | SSHD service | down for 2 minutes | | SUPERVISORD service | down for 2 minutes | | SMTP | if enabled, but stopped for 4 minutes | | NTPD | if enabled, but stopped for 4 minutes | | Any of the SUPERVISORD services | if it has crashed | | SCALE socket | not detected/does not exist for 4 minutes | | SCALE INPUT queue | receiving over 500 messages for 10 minutes | | SCALE RETRY queue | receiving over 100 messages for 10 minutes | | COLLECTOR queue | receiving over 1000 messages for 10 minutes | | CLASSIFICATION queue | receiving over 5000 messages for 10 minutes | In addition, the Manager sends syslog alerts for files that haven’t been processed in 24 hours and for file processing failures. ### Log Management **Applies to Spectra Detect Worker** Users can configure the [level](https://en.wikipedia.org/wiki/Syslog#Severity_level) of events that are sent to a syslog receiver (*Syslog log level*) or that are saved in internal logs (*TiScale log level*). Users cannot save high-severity events only but send lower-severity events to a syslog receiver. To send events, they first need to be saved, which is why the *TiScale log level* must always be equal or lower-severity than the value in *Syslog log level*. --- ## Spectra Detect Configuration — Appliances, Integrations, YARA, and Settings --- ## Spectra Detect Authentication — LDAP, OAuth 2.0, OpenID Connect, and SAML ## General authentication settings *Administration > Spectra Detect Manager > Authentication* ### Session, cookies and passwords - Duration of login session How long an authenticated user session will remain active on the appliance, set in seconds, minutes, hours or days. Minimum: 1 minute; maximum: 90 days. The default is 7 days. - Session Inactivity Timeout When selected, the session for every logged-in user will expire after the configured period of inactivity in seconds, minutes, hours or days. ------ The remainder of this section describes federated (single sign-on) login options. ## LDAP *Administration > Spectra Detect Manager > Authentication > User Directory: LDAP* ### Connection - LDAP server host Host name or IP address of the server providing LDAP authentication. Example: *ldap.example.com*. Click the *Test* button to verify the connection to the server. - LDAP server port LDAP server host port. Defaults: 389 (LDAP) or 636 (LDAPS). - TLS Select to use a TLS (secure) connection when communicating with the LDAP server. - TLS require certificate Select to require TLS certificate verification when communicating with the LDAP server. - Bind DN or user User to use when logging in to the LDAP server for searches. DN stands for Distinguished Name. Examples: *user@example.com* or *cn=user,dc=example,dc=com* - Password Password for the Bind user account. ### User Schema - Base DN Root node in LDAP from which to search for users. Example: *cn=users,dc=example,dc=com* - Scope Scope of the user directory searches (base, one level, subordinate, subtree). - User Object Class The objectClass value used for when searching users. Example: *user* - User Name Attribute The user name field. Examples: *sAMAccountName* or *cn* ### Group Schema **The majority of fields in this section are the same as in the User Schema section, except the settings relate to groups.** - Group Type LDAP group membership attribute (Member, Unique Member) ### User attribute mapping - First name Field to map to a user’s first name. Example: *givenName* - Last name Field to map to a user’s last name. Example: *sn* - E-mail Field to map to email. Example: *mail* ### User access - Active flag group Group DN. Users will be marked as active only if they belong to this group. Example: *cn=active,ou=users,dc=example,dc=com* - Superuser flag group Group DN. Users will be marked as superusers only if they belong to this group. Example: *cn=admins,ou=groups,dc=example,dc=com* - Require group Group DN. Authentication will fail for any user that does not belong to this group. Example: *cn=enabled,ou=groups,dc=example,dc=com* - Deny group Group DN. Authentication will fail for any user that belongs to this group. Example: *cn=disabled,ou=groups,dc=example,dc=com* ### Select TLS CA Certificate file - Select a file to upload The dialog that opens when clicking **Choose File** allows the user to upload their own TLS certificate for verifying the LDAP host identity. The certificate must be in PEM file format. To apply the certificate, the options *TLS* and *TLS require certificate* must be enabled. It is also possible to upload certificates through the Central Configuration Management section on Spectra Detect Manager, if the appliance is connected and authorized on the Manager. ## OAuth 2.0 / OpenID Connect 👉 Described in the [OpenID guide](/General/SecurityAndAccessControl/OpenID). ## SAML 👉 Described in the [SAML guide](/General/SecurityAndAccessControl/SAML). --- ## Spectra Detect Backup & Restore — Database and Configuration Management **Important: This section is available in OVA deployments.** ***Administration > Backup and Restore*** The Backup and Restore page allows administrators to create backups of the Spectra Detect Manager database and configuration, and restore them when needed. This feature is essential for disaster recovery, system migration, and maintaining data integrity. ## Backup Spectra Detect Manager Database and Configuration To create a backup of the Spectra Detect Manager database and configuration: 1. Navigate to **Administration > Backup and Restore**. 2. In the **Backup Spectra Detect Manager Database and Configuration** section, click **Generate Backup**. 3. The system creates a backup archive with a timestamped filename (e.g., `backup-spectra-detect-manager-2024/07/03-121916.bin`). 4. Once the backup is generated, the **Backup Archive** field displays the filename with a timestamp indicating when it was created. 5. Click the download icon next to the backup filename to download the backup archive to your local system. **Tip: Store backup archives in a secure location separate from the Spectra Detect Manager appliance to ensure data availability in case of system failure.** ## Restore Spectra Detect Manager Database and Configuration To restore a previously created backup: 1. Navigate to **Administration > Backup and Restore**. 2. In the **Restore Spectra Detect Manager Database and Configuration** section, under **Select Backup Archive**, click **Choose File**. 3. Browse to and select the backup archive file (`.bin` file) you want to restore. 4. The selected filename appears next to the **Choose File** button, along with a timestamp showing when the backup was generated. 5. Click **Restore** to begin the restoration process. 6. Confirm the restoration when prompted. **Warning: Restoring a backup will overwrite the current database and configuration settings. Ensure you have a recent backup of the current state before proceeding with a restore operation.** ## API The Backup and Restore page can also be managed using the Management API. For more information, see the [Management API guide](../API/ManagementAPI.md#back-up-and-restore-the-manager). ## Best Practices - **Regular backups**: Schedule regular backups to minimize data loss in case of system failure. - **Backup before updates**: Always create a backup before performing system updates or major configuration changes. - **Secure storage**: Store backup archives in a secure, off-site location with appropriate access controls. - **Test restores**: Periodically test the restore process to ensure backups are valid and can be successfully restored. - **Retention policy**: Maintain multiple backup versions and establish a retention policy based on your organization's requirements. --- ## Spectra Detect Certificate Management — Root CA Trust Store **Important: This section is only available in OVA deployments.** *Administration > Certificates* The **Root CA Trust Store Management** page enables administrators to manage Root CA certificates for Spectra Detect and [Spectra Analyze](/SpectraAnalyze/) appliances. Users can add, remove, trust, and distrust Root CA certificates through the web interface to customize the appliance's certificate validation behavior for secure communications. ## Certificate Scope Certificates managed through the Spectra Detect interface apply to: - **Spectra Detect Manager** - **Spectra Detect Hub** - **Spectra Detect Worker** - **Spectra Analyze** ## Accessing Certificate Management To access the certificate management interface: 1. Log into the appliance web interface as an administrator. 2. Navigate to **Administration > Certificates**. 3. The certificate management page displays all Root CA certificates currently in the Trust Store. ## Certificate List Overview The certificate management page displays a table with the following information for each certificate: | Column | Description | | -------------- | ------------------------------------------ | | **ID** | Number of certificate. | | **Subject** | Name of certificate. | | **Issuer** | Certificate issuer. | | **Valid From** | Certificate validity start date. | | **Valid To** | Certificate validity end date. | | **Filename** | Name of .pem file. | | **Trusted** | Whether the certificate is currently trusted. | | **Actions** | Available operations for the certificate. | ## Adding Root CA Certificates To add a new Root CA certificate to the Trust Store: 1. Under **Administration > Certificates**, click **Add Certificate**. 2. Click **Browse Files**, and select a valid certificate file in `.pem` format. 3. Click **Upload** to add the certificate to the Trust Store. 4. The certificate appears in the list with the **Trusted** status set to *Yes*. ## Managing Certificates To distrust or blocklist a certificate: 1. Locate the certificate in the list. 2. Under **Actions**, click **Distrust**. 3. Confirm the change in the modal dialog. 4. The **Trusted** status changes to *No*. To re-trust a certificate: 1. Locate the certificate in the list. 2. Under **Actions**, click **Trust**. 3. Confirm the change in the modal dialog. 4. The **Trusted** status changes to *Yes*. To remove a certificate: 1. Locate the certificate in the list. 2. Under **Actions**, click **Remove**. 3. Confirm the change in the modal dialog. 4. The certificate is deleted from the Trust Store. **Info: Some certificates may be greyed out because you don't have the necessary permissions to modify them.** To reset the Trust Store: 1. Click **Remove All Added Certificates**. 2. Confirm the action in the modal dialog. 3. All user-added certificates are removed from the Trust Store. **Warning: Removal of certificates works for both trusted and distrusted certificates and completely deletes them from the system. ** Clicking **Remove All Added Certificates** removes all certificates added by users and cannot be undone. --- ## Spectra Detect Email Alerting — Quota Notifications and Alert Thresholds **Important: This section is only available in OVA deployments.** The Email Alerting page allows users to configure Spectra Intelligence quota usage notifications. Users can enable email alerting, manage recipients, and define alert thresholds based on system quotas. To receive email alerts, users must configure Spectra Intelligence via **Central Configuration**. - **Enable Email Alerting**: Select this checkbox to enable sending system alerts via email. Once enabled, any alerts generated by the system, such as quota warnings or system issues, will be distributed to the specified recipients. - **Add All Users to the Recipients List**: Select this checkbox to add all system users to the recipients list for email alerts. - **Recipients**: Enter the email addresses of the individuals who should receive system alerts. Multiple email addresses can be added by pressing the Enter key after each one. If the **Add All Users to the Recipients List** option is checked, additional recipients cannot be manually added. - **Alert when quota has reached**: Choose the quota level at which email alerts will be triggered. You can select one or more thresholds. The available options are: `50%`, `75%`, and `90%`. - **Alert when quota has been exceeded**: Select this option to trigger an alert when the system's quota has been exceeded. --- ## Spectra Detect Filter Management — Advanced Egress Filtering Rules **Important: This section is only available in OVA deployments.** In addition to regular [file filtering on egress integrations](../Config/ApplianceConfiguration.md#file-filters), you can set up an advanced filter. Advanced filters have more options and therefore allow more granularity. **Note: Advanced filters are currently available for [Spectra Analyze](/SpectraAnalyze/) Integration (Central Configuration > Egress Integrations > Spectra Analyze Integration), AWS S3 (Central Configuration > Egress Integrations > AWS S3 > File Storage/Unpacked Files Storage/Report Storage > Enable Filter), and Callback (Central Configuration > Egress Integrations > Callback > Enable Filter)** ### Types of filters Filters can be either **inclusive** (Pass Files) or **exclusive** (Drop Files). If a filter is inclusive, that means that all files that match your criteria will be saved to external storage. If it's exclusive, you will save everything *except* the files that match your criteria. When creating a filter, you can also specify that it should only apply to top-level parent files: **Filter Applies Only to Container**. In this case, extracted files will not be saved. ### Conditions Conditions allow you to specify rules based on various attributes such as file type, file size, classification, and more. Conditions are grouped into categories: - **File**: filters related to file names, recognized file types, and file size - **Classification**: filters related to [how we classify files](/General/AnalysisAndClassification/Classification) - **Identification**: filters for file format identification - **Behavior**: filters for network calls made by a file (see the [report schema](/SpectraDetect/report-schema#uri)) - **Document**: if a file is identified as a document, these filters allow you to narrow down what you save based on the document attributes, such as number of pages or word count - **Unpacking**: similar to how the "Identification" category of conditions relates to file format identification, this category contains filters based on file unpacking - **File statistics**: filters operating on statistics produced by an analysis (for example, the `count` of files of a particular type found in an analyzed file) - **Capabilities**: different capabilities detected in the analyzed file - **YARA matches**: see description of the filters [in the report schema](/SpectraDetect/report-schema/#yara) - **Indicators**: see [report schema](/SpectraDetect/report-schema/#indicator) - **Mitre**: one of the Mitre [techniques](https://attack.mitre.org/techniques/enterprise/) - **Tags**: the full list is available in the [report schema appendix](/SpectraDetect/report-schema/#appendix-c-spectra-core-tags) --- ## Spectra Detect Licensing — Activation, Trial Period, and Compliance **Important: This section is only available in OVA deployments.** ***Administration > Licensing*** On first login after installing or updating the appliance, the appliance must be licensed within 45 days of the release’s general availability date. This also applies to any connected appliances. While the trial license is active, Licensing options on the Manager can be accessed using the **Administration > Licensing** menu item. There are two ways of licensing appliances: ### By using Spectra Intelligence Click the **Activate Using Spectra Intelligence** button and fill out the account information. A licensing request will be sent to Spectra Intelligence and, if the account is valid, the appliance will be activated. Individual appliances connected to the Manager can be activated using Spectra Intelligence by configuring it for appliance groups in **Central Configuration**. ### By uploading a license file Appliances can also be licensed offline by sending their machine IDs to ReversingLabs support via email. This can be performed from the licensing page by checking one or more boxes next to appliances and clicking the **Request License** button. This opens the user’s default email client with the relevant information filled in. Make sure to send the request using an email address that is previously known to ReversingLabs. When we respond with the requested license files, upload them using the **Upload License** button and click **Upload**. The Manager will automatically match the license files to appropriate appliances. A single license file can contain multiple machine IDs. **If an appliance instance was created by cloning a VM, administrators need to generate a new Machine ID and request a new license for every clone of the original appliance VM.** If the appliance is still in the licensing trial period, this can be done in the **Administration > Licensing** section. **License Expiration** - Appliances without a license are in a trial period for 45 days from the release’s general availability date. - If appliances licensed using Spectra Intelligence can’t reach it, they enter a grace period of 14 days during which they will still operate normally. - Regenerating a machine ID of an already licensed appliance will require it to be licensed again. - Once the Manager trial/grace period expires, the appliance will open to the Licensing screen, and no other actions will be available. **Note: Licensing can also be configured using the Spectra Detect Manager API. Visit **Help > Spectra Detect Manager API Documentation** for more information. To license Spectra Detect appliances without using the Manager, refer to the API section of the Spectra Detect user guide.** --- ## Spectra Detect Logs — LogQL Query and Search **Important: This section is only available in Kubernetes microservices deployments.** ***Administration > Logs*** The Logs page provides a centralized interface for searching, filtering, and analyzing logs from all Spectra Detect microservices using LogQL (Log Query Language). This feature enables administrators to troubleshoot issues, monitor system behavior, and audit operations across the deployment. ## Accessing Logs Navigate to **Administration > Logs** to access the log search interface. ## Query Interface The Logs page includes the following components: - **LogQL Query Field**: Enter LogQL queries to filter and search logs. The default query is `{namespace="default"} |= "error"`, which displays all error-level logs from the default namespace. - **Time Range Selector**: Select the time range for log retrieval. Options include: - Last Hour (default) - Custom time ranges - **Log Type Filter**: Filter logs by severity level: - All Types (default) - Debug - Error - Info - Unknown - Warn - **Run Query Button**: Execute the LogQL query to retrieve matching logs. - **Export Button**: Export the current log results as a CSV file for offline analysis or archival. ## Log Display The log results are displayed in a table format with the following columns: - **Time**: Timestamp of the log entry in UTC format. - **Namespace**: The Kubernetes namespace where the log originated (e.g., `default`). - **Pod**: The pod name that generated the log entry (e.g., `sdm-micro-sdm-portal-0`). - **Type**: The log severity level (Debug, Error, Info, Unknown, Warn). - **Message**: The full log message content. ### Log Severity Indicators Log counts by severity level are displayed above the results table: - **Debug**: Development and diagnostic information - **Error**: Error conditions requiring attention - **Info**: Informational messages about normal operations - **Unknown**: Logs without a recognized severity level - **Warn**: Warning conditions that may require investigation ## Using LogQL Queries LogQL is the query language used to filter and search logs. For complete LogQL syntax and query examples, see the [Grafana Loki LogQL documentation](https://grafana.com/docs/loki/latest/query/). **Note: Only log queries are supported. Metric queries are not supported in this interface.** ## Exporting Logs To export log results: 1. Run your LogQL query to retrieve the desired logs. 2. Click the **Export** button in the top-right corner. 3. The logs will be downloaded as a CSV file for offline analysis. ## API Access The Logs feature also provides API endpoints for programmatic access to log data. For API documentation and usage examples, navigate to **Help > API Documentation** in the Spectra Detect Manager interface. --- ## Spectra Detect Manager Settings — Network, SSL, and System Configuration These are the general steps to configuring a new Manager: 1. Deploy the appliance and attach it to the network. 2. Configure network settings via the console to access the Web UI. 3. Configure installation-specific settings on the system configuration screen. 4. [License](Licensing.md) the Manager ## Network Ports The Manager supports the following ports for inbound connections: - 80/TCP and 443/TCP for connecting to the Manager Web UI. - 22/TCP for maintenance purposes. - 161/UDP for SNMP monitoring Outgoing connections to the internet via the following ports are also supported: - 53/UDP for DNS - 123/UDP for NTP However, it is strongly recommended that the users configure the system to use their own DNS and NTP infrastructure (if necessary). For outgoing connections to the [Spectra Intelligence](/SpectraIntelligence/) database at `https://appliance-api.reversinglabs.com`, the destination port is 443/TCP. The DNS name is `appliance-api.reversinglabs.com` and the connection supports HTTPS only. ## Configuration via the Manager Web Interface After logging in, access the **Administration ‣ Spectra Detect Manager** page from the main Manager menu. The page contains dialogs with options for configuring the Manager. When done updating the settings in the configuration dialogs, click **Save**. The appliance will be restarted and begin using the new settings. ### General | | | | ----------------------------------- | ------------------------------------------------------------ | | **Network settings** | | | Application URL | The URL used to access the Web UI of the Manager. **The application URL must be configured to use the HTTPS protocol.** | | Allowed hosts | A list of strings, one per line, representing the host/domain names that this appliance installation can serve. Values in this list can be fully qualified names (e.g., "www.example.com"), in which case they will be matched against the request’s host header exactly (case-insensitive, not including port). A value beginning with a period can be used as a subdomain wildcard: ".example.com" will match "example.com", "www.example.com", and any other subdomain of "example.com". A value of "*" will match anything. Examples: *.reversinglabs.com, 89.201.174.154, 89.201.174.152*. **If the Spectra Detect Manager is configured as a redundant cluster, this list must include the hostname of the proxy server.** | | Select SSL certificate | Clicking **Browse** allows the user to upload a file containing a custom SSL certificate to replace the self-signed certificate generated by the Manager. | | Select SSL certificate key | Clicking **Browse** allows the user to upload a file containing the key that corresponds to the certificate uploaded above. | | **Synchronization** | | | Enable YARA ruleset synchronization | Select the checkbox to allow synchronizing YARA rulesets between the appliances connected to the Manager. This global setting affects all Spectra Analyze and Spectra Detect Worker appliances. For this functionality to work, YARA synchronization must also be enabled on connected Spectra Analyze appliances. See the [YARA Sync Page](./YARASync.md) section for more details. | | **SSH** | | | Permit root SSH login | Select the checkbox to allow root SSH access to the Manager. This setting can be used for automated password management. | | **SWAP** | | | Disable SWAP memory | Checking this option will disable the usage of SWAP memory. Not applicable if appliance is deployed as a Docker image. Enabled by default. | ### SMTP | | | | ---------------------------- | ------------------------------------------------------------ | | SMTP hostname | The host to use for sending email. For the SMTP service to function properly, this field must not be empty. | | SMTP port | Port of the host used for sending email. For the SMTP service to function properly, this field must not be empty. | | Username; Password | SMTP username and password for authentication. | | Default "from" email address | The email address used by the appliance as the "from" address when sending email (for password resets, error alerts…). | | Use TLS | Select the checkbox to use a secure connection (TLS; Transport Layer Security) when communicating with the SMTP server. | ### SNMP & system alerting | | | | --------------------------------------------- | ------------------------------------------------------------ | | Enable SNMP service | Select the checkbox to enable Simple Network Management Protocol service. | | Community | Enter the name of an SNMP community list for authentication. Community is a list of SNMP clients authorized to make requests. The SNMP service will not function properly if this field is not configured. | | Enable trap sink | Select the checkbox to enable sending SNMP traps to the sink server. Traps are asynchronous, unsolicited SNMP messages sent by the SNMP agent to notify about important events on the appliances. | | Trap community | Enter the SNMP trap community string. If the *Enable SNMP service* and *Enable trap sink* checkboxes are selected, then this field is required. | | Trap sink server | Enter the host name or the IP address of the trap sink server. The sink server is the location to which SNMP traps will be sent. If the *Enable SNMP service* and *Enable trap sink* checkboxes are selected, then this field is required. | | SNMP trap thresholds | A set of configuration fields allowing the user to set the thresholds (values that will trigger an SNMP trap) for supported types of events. Thresholds can be configured for average system load in 1, 5, and 10 minutes (as a percentage), used memory and used disk space (as a percentage). | | **System Alerting** | | | Send system alert messages to syslog server | Select the checkbox to enable sending alerts about the status of critical system services on the connected appliances to the syslog server. | | Host | Host address of the remote syslog server to send alerts to. | | Port | Port of the remote syslog server. | | Protocol | Communication protocol to use when sending alerts to a remote syslog server. Options are TCP (default) and UDP. | | Enable audit logs to be sent to syslog server | Audit logs will be automatically sent to the syslog server in addition to other system messages. Enabling this will increase the traffic between the Manager and the syslog server. | #### Spectra Detect MIB Descriptions | MIB Module | Value | OID | Description | |----------------------|------------------------------|--------------------------------------|------------------------------------------------------------------| | RL-MIB | `device` | .1.3.6.1.4.1.48699.1.1 | | | RL-MIB | `tiscalehub` | .1.3.6.1.4.1.48699.1.1.6 | | | RL-MIB | `c1000` | .1.3.6.1.4.1.48699.1.1.3 | | | RL-MIB | `tiscaleworker` | .1.3.6.1.4.1.48699.1.1.5 | | | RL-TISCALEHUB-MIB | `tshMib` | .1.3.6.1.4.1.48699.1.1.6.1 | | | RL-TISCALEHUB-MIB | `tshMibObjects` | .1.3.6.1.4.1.48699.1.1.6.1.1 | | | RL-TISCALEHUB-MIB | `tshScalars` | .1.3.6.1.4.1.48699.1.1.6.1.1.1 | | | RL-TISCALEHUB-MIB | `tshVersionString` | .1.3.6.1.4.1.48699.1.1.6.1.1.1.1 | Product version string. | | RL-TISCALEHUB-MIB | `tshQueues` | .1.3.6.1.4.1.48699.1.1.6.1.1.2 | | | RL-TISCALEHUB-MIB | `tshPostFixQueue` | .1.3.6.1.4.1.48699.1.1.6.1.1.2.1 | | | RL-TISCALEHUB-MIB | `tshPFQueueName` | .1.3.6.1.4.1.48699.1.1.6.1.1.2.1.1 | Queue name. | | RL-TISCALEHUB-MIB | `tshPFQueueSize` | .1.3.6.1.4.1.48699.1.1.6.1.1.2.1.2 | Queue size. | | RL-TISCALEHUB-MIB | `tshPFQueueStatus` | .1.3.6.1.4.1.48699.1.1.6.1.1.2.1.3 | Queue status. | | RL-TISCALEHUB-MIB | `tshMibNotifications` | .1.3.6.1.4.1.48699.1.1.6.1.2 | | | RL-TISCALEHUB-MIB | `tshPFQueueThreshold` | .1.3.6.1.4.1.48699.1.1.6.1.2.1 | Queue size status based on configured threshold. | | RL-TISCALEHUB-MIB | `tshMibConformance` | .1.3.6.1.4.1.48699.1.1.6.1.3 | | | RL-TISCALEWORKER-MIB | `tswMib` | .1.3.6.1.4.1.48699.1.1.5.1 | | | RL-TISCALEWORKER-MIB | `tswMibObjects` | .1.3.6.1.4.1.48699.1.1.5.1.1 | | | RL-TISCALEWORKER-MIB | `tswScalars` | .1.3.6.1.4.1.48699.1.1.5.1.1.1 | | | RL-TISCALEWORKER-MIB | `tswVersionString` | .1.3.6.1.4.1.48699.1.1.5.1.1.1.1 | Product version string. | | RL-TISCALEWORKER-MIB | `tswQueueLength` | .1.3.6.1.4.1.48699.1.1.5.1.1.1.2 | Current input queue length. | | RL-TISCALEWORKER-MIB | `tswTables` | .1.3.6.1.4.1.48699.1.1.5.1.1.2 | | | RL-TISCALEWORKER-MIB | `tswStatsNumber` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.1 | The number of entries in tswStatsTable. | | RL-TISCALEWORKER-MIB | `tswStatsTable` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.2 | A list of statistics entries. | | RL-TISCALEWORKER-MIB | `tswStatsEntry` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.2.1 | An entry containing statistics information. | | RL-TISCALEWORKER-MIB | `tswStatsIndex` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.2.1.1 | A unique value, greater than zero, for each statistics interval. | | RL-TISCALEWORKER-MIB | `tswStatsDesc` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.2.1.2 | Description of a statistics interval. | | RL-TISCALEWORKER-MIB | `tswStatsFilesSubmitted` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.2.1.3 | Number of files submitted for processing. | | RL-TISCALEWORKER-MIB | `tswStatsFilesProcessed` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.2.1.4 | Number of files sucessfully processed. | | RL-TISCALEWORKER-MIB | `tswStatsBytesSubmitted` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.2.1.5 | Total size of files submitted for processing. | | RL-TISCALEWORKER-MIB | `tswStatsBytesProcessed` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.2.1.6 | Total size of files sucessfully processed. | | RL-TISCALEWORKER-MIB | `tswStatsFilesUnpacked` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.2.1.7 | Number of files unpacked. | | RL-TISCALEWORKER-MIB | `tswStatsBytesUnpacked` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.2.1.8 | Total size of files unpacked. | | RL-TISCALEWORKER-MIB | `tswStatsMeanQueueSize` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.2.1.9 | Mean input queue size. | | RL-TISCALEWORKER-MIB | `tswQueueNumber` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.3 | The number of entries in tswQueueTable. | | RL-TISCALEWORKER-MIB | `tswQueueTable` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.4 | A list of queues. | | RL-TISCALEWORKER-MIB | `tswQueueEntry` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.4.1 | An entry containing queue information. | | RL-TISCALEWORKER-MIB | `tswQueueIndex` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.4.1.1 | A unique value, greater than zero, for each queue. | | RL-TISCALEWORKER-MIB | `tswQueueName` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.4.1.2 | Queue name. | | RL-TISCALEWORKER-MIB | `tswQueueSize` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.4.1.3 | Queue size. | | RL-TISCALEWORKER-MIB | `tswQueueStatus` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.4.1.4 | Queue status. | | RL-TISCALEWORKER-MIB | `tswCPUNumber` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.5 | The number of entries in tswCPUTable. | | RL-TISCALEWORKER-MIB | `tswCPUTable` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.6 | A list of CPU entries. | | RL-TISCALEWORKER-MIB | `tswCPUEntry` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.6.1 | An entry containing CPU information. | | RL-TISCALEWORKER-MIB | `tswCPUIndex` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.6.1.1 | A unique value, greater than zero, for each CPU entry. | | RL-TISCALEWORKER-MIB | `tswCPUCores` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.6.1.2 | Number of CPU cores. | | RL-TISCALEWORKER-MIB | `tswCPUCoresStatus` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.6.1.3 | Status of CPU cores. | | RL-TISCALEWORKER-MIB | `tswCPUUtilization` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.6.1.4 | CPU utilization. | | RL-TISCALEWORKER-MIB | `tswCPUUtilizationStatus` | .1.3.6.1.4.1.48699.1.1.5.1.1.2.6.1.5 | CPU utilization status. | | RL-TISCALEWORKER-MIB | `tswMibNotifications` | .1.3.6.1.4.1.48699.1.1.5.1.2 | | | RL-TISCALEWORKER-MIB | `tswQueueThreshold` | .1.3.6.1.4.1.48699.1.1.5.1.2.1 | Queue size status based on configured threshold. | | RL-TISCALEWORKER-MIB | `tswCPUCoreThreshold` | .1.3.6.1.4.1.48699.1.1.5.1.2.2 | CPU core status change notification. | | RL-TISCALEWORKER-MIB | `tswCPUUtilizationThreshold` | .1.3.6.1.4.1.48699.1.1.5.1.2.3 | CPU utilization status change notification. | | RL-TISCALEWORKER-MIB | `tswMibConformance` | .1.3.6.1.4.1.48699.1.1.5.1.3 | | | RL-CM-MIB | `cmMib` | .1.3.6.1.4.1.48699.1.1.3.1 | | | RL-CM-MIB | `cmMibObjects` | .1.3.6.1.4.1.48699.1.1.3.1.1 | | | RL-CM-MIB | `cmScalars` | .1.3.6.1.4.1.48699.1.1.3.1.1.1 | | | RL-CM-MIB | `cmMibNotifications` | .1.3.6.1.4.1.48699.1.1.3.1.2 | | | RL-CM-MIB | `redundancyTrigger` | .1.3.6.1.4.1.48699.1.1.3.1.2.2 | Failover on HA system | | RL-CM-MIB | `redundancyTriggerOk` | .1.3.6.1.4.1.48699.1.1.3.1.2.3 | HA System resumed operation | | RL-CM-MIB | `cmMibConformance` | .1.3.6.1.4.1.48699.1.1.3.1.3 | | Download the MIB file in CSV format. ### Authentication See the [Authentication](Authentication.md) section. ### Spectra Intelligence | | | | ------------------------------ | ------------------------------------------------------------ | | Enable Spectra Intelligence | Select the checkbox to enable the connection to Spectra Intelligence. Spectra Detect Manager needs to be connected to the Spectra Intelligence cloud in order to automatically retrieve system updates and appliance upgrades. When connected, the Manager polls the cloud once every 60 minutes. | | Username; Password | Username and password for authenticating to Spectra Intelligence. | | Timeout | Specify how long to wait before the Spectra Intelligence connection times out (in seconds; the maximum allowed value is 1000). | | Proxy host | Proxy hostname for routing requests from the appliance to Spectra Intelligence (e.g., 192.168.1.15). | | Proxy port | Proxy port number (e.g., 1080). | | Proxy username; Proxy password | Username and password for proxy authentication. | ### Dashboard configuration - Enable Central Logging Enabling central logging will completely change the home page to show statistics on the number of processed files and their classifications. This feature is also resource-intensive. Ensure at least 32 GB RAM and 1 TB disk for optimal performance. - Retention period How long to keep the collected logs on the Manager. - Enable Central File Storage Enables file storage on the Manager. If enabled, connected Workers will store samples on the Manager. Stored samples can later be analyzed with Spectra Analyze by clicking on "Analyze with Spectra Analyze" on the analytics page. Enabling this feature may require additional disk space. The required storage depends on the size of the samples coming from the connected Workers and their retention period. Samples larger than the file limit threshold will not be stored. - File Size Limit File size limit in MiB. Samples larger than the set threshold will not be stored. The default is 400, the maximum supported file size on Spectra Analyze. - Sample Retention Period Time, in hours, after which the uploaded samples will be removed from the Central File Storage. - Minimum Disk Space The minimum allowed free disk space in GiB. If the remaining disk space is below the configured threshold, new sample uploads will be rejected. For example, to use 900 GiB of space for central file storage on a 1000 GiB disk, set the value to 100. - Enable Deep Cloud Analysis Enabling Multi-Scanning instructs Workers to upload samples to the Cloud using their respective account and usage quota. Samples are uploaded only if they pass the filtering criteria: up to 2GB in size. If a sample already exists in the Cloud, the Manager monitors data changes in the data change feed and updates the dashboard accordingly. Enabling this feature impacts the final verdict - classification, risk score and threat name, resulting in increased detection rate and reduced remediation time. Additionally, up to 5 antivirus engine scanners can be selected to be listed on the dashboard. Deep Cloud Analysis parameters can be configured in the **Central Configuration > [Spectra Intelligence](../Config/ApplianceConfiguration.md#spectra-intelligence)** section under the **Deep Cloud Analysis** tab. For more information on the Deep Cloud Analysis pipeline, refer to the [Deep Cloud Analysis Pipeline](../Usage/Analysis.md#deep-cloud-analysis-pipeline-diagram) diagram. - Subscribe to Cloud Classification Changes Receive notifications about cloud classification changes from Spectra Intelligence directly on the Spectra Detect Dashboard. To configure additional alerts via UI, email, splunk, or syslog, navigate to the [Notifications](../Usage/Notifications.md) section. Even if the *Scan Unpacked Files* option in the [Spectra Intelligence configuration](../Config/ApplianceConfiguration.md#spectra-intelligence) section is enabled, the notifications about cloud classification changes will be received only for top-level containers. ### System time - Enable network time synchronization Select the checkbox to enable clock synchronization via NTP (Network Time Protocol). - Timezone Select the timezone from the dropdown list. - NTP servers A list of server addresses, separated by a new line, to use for system clock synchronization. Click **Test connection** to verify that time synchronization functions properly. ### System Alerting If system alerting is enabled in the *System Alerting* configuration dialog, the following system operations and services will be monitored. Syslog notifications are sent when any of the services or operations meet the condition(s) defined in the table. | SYSTEM OPERATION OR SERVICE | NOTIFICATION TRIGGER | | ------------------------------- | ------------------------------------- | | RAM | usage is over 90% for 10 minutes | | CPU | usage is over 40% for 2 minutes | | CPU wait (waiting for IO) | over 20% for 2 minutes | | Disk usage | over 90% for 10 minutes | | UWSGI service | down for 2 minutes | | NGINX service | down for 2 minutes | | RABBIT-MQ service | down for 2 minutes | | POSTGRES service | down for 2 minutes | | MEMCACHED service | down for 2 minutes | | CROND service | down for 2 minutes | | SSHD service | down for 2 minutes | | SUPERVISORD service | down for 2 minutes | | SMTP | if enabled, but stopped for 4 minutes | | NTPD | if enabled, but stopped for 4 minutes | | Any of the SUPERVISORD services | if it has crashed | #### SNMP Trap Thresholds The Manager can receive notifications (traps) about important system events via the Simple Network Management Protocol (SNMP). The events are "trapped" and sent to the trap sink server when their configured threshold levels are triggered. The Manager uses the **DISMAN-EVENT-MIB::mteTriggerFired** SNMP trap and supports 3 different triggers. These triggers can be used to keep track of low disk space, high memory usage or high CPU load average over time. | TRIGGER IDENTIFIER | TRIGGER CONDITION | | -------------------------------------------------------- | ------------------------------------------------------------ | | `DISMAN-EVENT-MIB::mteHotTrigger.0 = STRING: dskTable` | disk usage is higher than the configured threshold (the default value is 90%) | | `DISMAN-EVENT-MIB::mteHotTrigger.0 = STRING: memoryFree` | memory usage is higher than the set threshold (the default value is 80%) | | `DISMAN-EVENT-MIB::mteHotTrigger.0 = STRING: laTable` | average system load in the specified time frame (1, 5 or 15 minutes) is higher than the set threshold | To enable SNMP traps and configure the address of the trap sink server, adjust the values in the **Settings ‣ Configuration ‣ SNMP & System Alerting** dialog on the Manager. The dialog also allows setting thresholds for supported types of events, which are described in more detail below. **Average system load** This trap is sent if the average load of the local system exceeds specified values (1-minute, 5-minute and 15-minute averages). Values should be provided as percentages, which are recalculated into appropriate thresholds as reported with uptime or top commands. The following examples show traps triggered by a high 1-minute, 5-minute and 15-minute system load average, respectively: ``` 2018-01-26 14:35:54 [UDP: [192.168.123.247]:60418->[192.168.123.17]:162]: DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (13) 0:00:00.13 SNMPv2-MIB::snmpTrapOID.0 = OID: DISMAN-EVENT-MIB::mteTriggerFired DISMAN-EVENT-MIB::mteHotTrigger.0 = STRING: laTable DISMAN-EVENT-MIB::mteHotTargetName.0 = STRING: DISMAN-EVENT-MIB::mteHotContextName.0 = STRING: DISMAN-EVENT-MIB::mteHotOID.0 = OID: UCD-SNMP-MIB::laErrorFlag.1 DISMAN-EVENT-MIB::mteHotValue.0 = INTEGER: 1 UCD-SNMP-MIB::laNames.1 = STRING: Load-1 UCD-SNMP-MIB::laErrMessage.1 = STRING: 1 min Load Average too high (= 2.56) ``` ``` 2018-01-26 14:35:54 [UDP: [192.168.123.247]:60418->[192.168.123.17]:162]: DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (13) 0:00:00.13 SNMPv2-MIB::snmpTrapOID.0 = OID: DISMAN-EVENT-MIB::mteTriggerFired DISMAN-EVENT-MIB::mteHotTrigger.0 = STRING: laTable DISMAN-EVENT-MIB::mteHotTargetName.0 = STRING: DISMAN-EVENT-MIB::mteHotContextName.0 = STRING: DISMAN-EVENT-MIB::mteHotOID.0 = OID: UCD-SNMP-MIB::laErrorFlag.2 DISMAN-EVENT-MIB::mteHotValue.0 = INTEGER: 1 UCD-SNMP-MIB::laNames.2 = STRING: Load-5 UCD-SNMP-MIB::laErrMessage.2 = STRING: 5 min Load Average too high (= 2.00) ``` ``` 2018-01-26 14:35:54 [UDP: [192.168.123.247]:60418->[192.168.123.17]:162]: DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (13) 0:00:00.13 SNMPv2-MIB::snmpTrapOID.0 = OID: DISMAN-EVENT-MIB::mteTriggerFired DISMAN-EVENT-MIB::mteHotTrigger.0 = STRING: laTable DISMAN-EVENT-MIB::mteHotTargetName.0 = STRING: DISMAN-EVENT-MIB::mteHotContextName.0 = STRING: DISMAN-EVENT-MIB::mteHotOID.0 = OID: UCD-SNMP-MIB::laErrorFlag.3 DISMAN-EVENT-MIB::mteHotValue.0 = INTEGER: 1 UCD-SNMP-MIB::laNames.3 = STRING: Load-15 UCD-SNMP-MIB::laErrMessage.3 = STRING: 15 min Load Average too high (= 2.05) ``` **Used memory** This trap is sent if used memory on the local system exceeds the specified percentage. The default value is 80%. The following example shows an event triggered by memory usage that exceeded the configured trap threshold: ``` DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (8) 0:00:00.08 SNMPv2-MIB::snmpTrapOID.0 = OID: DISMAN-EVENT-MIB::mteTriggerFired DISMAN-EVENT-MIB::mteHotTrigger.0 = STRING: memoryFree DISMAN-EVENT-MIB::mteHotTargetName.0 = STRING: DISMAN-EVENT-MIB::mteHotContextName.0 = STRING: DISMAN-EVENT-MIB::mteHotOID.0 = OID: UCD-SNMP-MIB::memTotalFree.0 DISMAN-EVENT-MIB::mteHotValue.0 = INTEGER: 2124816 UCD-SNMP-MIB::memTotalReal.0 = INTEGER: 16467096 kB ``` **Used disk space** This trap is sent if used disk space on any of the mounted disks exceeds the specified percentage. The default value is 90%. The following example shows an event triggered by a disk with less than 10% of free disk space on the /boot partition: ``` DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (25) 0:00:00.25 SNMPv2-MIB::snmpTrapOID.0 = OID: DISMAN-EVENT-MIB::mteTriggerFired DISMAN-EVENT-MIB::mteHotTrigger.0 = STRING: dskTable DISMAN-EVENT-MIB::mteHotTargetName.0 = STRING: DISMAN-EVENT-MIB::mteHotContextName.0 = STRING: DISMAN-EVENT-MIB::mteHotOID.0 = OID: UCD-SNMP-MIB::dskErrorFlag.26 DISMAN-EVENT-MIB::mteHotValue.0 = INTEGER: 1 UCD-SNMP-MIB::dskPath.26 = STRING: /boot UCD-SNMP-MIB::dskErrorMsg.26 = STRING: /boot: less than 10% free (= 8%) ``` --- ## Spectra Detect Redundancy — Manager Clustering and Failover **Important: This section is only available in OVA deployments.** Spectra appliances are designed for full redundancy. If one component fails, another automatically takes over using the same configuration settings. This chapter describes how redundancy works for **Spectra Detect Manager**. ## Creating Manager Redundancy **Important: A proxy or Load balancer is REQUIRED for Spectra Detect Manager to function properly in Redundancy.** Use the following endpoint to identify the active/primary Spectra Detect Manager: `GET /api/v1/health-check/` This endpoint returns a `200` status code if the cluster node is primary and up. It returns a `503` status code if the cluster node is secondary or down. The endpoint does not provide a response body. Before clustering, add the active/primary manager (the one you will use to initiate redundancy) to the proxy or load balancer. After redundancy setup completes, update the proxy to include both managers (this is described in [step 7](#7-update-the-proxy-or-load-balancer)). ### 1. Configure Allowed Hosts As admin, go to [Administration > Spectra Detect Manager > General > Network Settings](../Admin/ManagerSettings.md#general). - Verify that the **Application URL** is set to the proxy IP or domain. - In **Allowed Hosts**, ensure the following are listed: 1. Current Detect Manager IP address and domain 2. Proxy IP and domain 3. Secondary IP and domain 4. localhost and 127.0.0.1 ![Allowed Hosts](./images/redundancy_1.png) - Click **Save** on the primary manager. - Repeat for the secondary manager, but **only update the Allowed Hosts section**. Do not modify the Application URL on the secondary manager. It will be automatically replaced during clustering. **Note:** Changing the Application URL updates the configuration on all connected appliances (Hub, Workers, and Analyze). ### 2. Create Redundant Cluster Go to **Administration > Redundancy**, then click **Create redundant cluster**. ![Create Redundant Cluster](./images/redundancy_2.png) ### 3. Establish Connection ![Establish Connection](./images/redundancy_3.png) - Fill out the required fields. - A VPN is automatically established between redundant managers. - The user on the secondary manager must be an admin. - Set **Failover Timeout (seconds)** between 3 and 600; 30 is recommended. - Usually, the machines in a redundant cluster share the same TLS certificate. However, if you need individual TLS certificates for each machine, you can *Disable TLS Certificate Sharing*. - Click **Next**. ### 4. Check Prerequisites ![Check Prerequisites](./images/redundancy_4.png) - The system validates if your environment supports redundancy. - When all checks pass, the message "All checks successfully passed" appears. - Click **Next** to continue. ### 5. Run the Configuration Process ![Run the Configuration Process](./images/redundancy_5.png) - Click **Start Configuration**, then confirm in the popup. - **Do not refresh or navigate away** during setup; leaving the page interrupts the process and it cannot be reopened. - The system automatically switches to the maintenance screen while redundancy is configured. - When setup completes, both managers reboot and return online automatically. Wait until both are reachable before proceeding. **Important: When initiating redundancy on a Spectra Detect Manager with Central logging enabled and a large volume of stored sample data, the initial sync to the secondary manager may take longer than 30 minutes.** If you refresh the screen after 30 minutes, you may see a "Configuration failed" status with a "Rollback Configuration" button, even though clustering is still in progress. Do not roll back while data is syncing. To verify sync progress, SSH to each SDM and run the `top` command. If the `rsync` process is using ~70% CPU or more, the sync is still in progress. Wait until the `rsync` process on both the primary and secondary is under ~10% CPU before refreshing the GUI. - When the "Configuration finished successfully" message appears, click **Finish**. ![Finish the Configuration Process](./images/redundancy_6.png) ### 6. Verify Status - Open the **Cluster Configuration** tab to see which manager is primary and which is secondary. - The secondary manager is read-only while redundancy is active. - The **Status** tab provides redundancy health and logs. - You can initiate a switchover or remove the cluster configuration here. - Check the [Manage and View the Status of Redundancy](#manage-and-view-the-status-of-redundancy) section for more details. ### 7. Update the Proxy or Load Balancer - After the redundancy setup is complete, update the proxy or load balancer to include both manager IPs. - Access the Spectra Detect GUI via the proxy or load balancer; it automatically directs users to the primary node. - When you later remove redundancy, follow the guidance in the [Post De-clustering Actions](#post-de-clustering-actions) section below to keep only one manager active. ## Manage and View the Status of Redundancy ### Redundancy Status Go to **Administration > Redundancy > Cluster Configuration**. ![Cluster Configuration](./images/redundancy_7.png) - The **Cluster Configuration** tab shows which manager is primary or secondary. - Click the **Status** tab for detailed health checks and logs. - Three log views are available: RL Cluster, RL Daemon, and Corosync. ![Cluster Status](./images/redundancy_8.png) ### Managing Switchovers ![Manual Switchover](./images/redundancy_9.png) 1. Go to **Administration > Redundancy > Cluster Configuration**. 2. Click **Manual Switchover** to make the secondary manager primary. 3. Confirm the action when prompted. ![Manual Switchover](./images/redundancy_10.png) 4. The system enters maintenance mode for several minutes. 5. When complete, the proxy or load balancer automatically directs traffic to the new primary. ## Upgrading Redundant Managers **Important: The primary manager does not automatically upgrade the secondary manager. Upgrades must be performed manually to give administrators full control over upgrade sequencing and minimize service disruption.** To upgrade redundant Spectra Detect Managers: 1. **Upgrade the secondary manager** - The secondary manager can be upgraded while the primary continues to serve traffic - Follow the standard upgrade procedure for the secondary manager - Wait for the upgrade to complete and the secondary to come back online 2. **Perform a manual switchover** - Go to **Administration > Redundancy > Cluster Configuration** - Click **Manual Switchover** to make the upgraded secondary the new primary - The proxy or load balancer will automatically direct traffic to the new primary 3. **Upgrade the new secondary manager** - The former primary (now secondary) can be upgraded - Follow the standard upgrade procedure - Wait for the upgrade to complete Both managers are now upgraded and the cluster is fully operational. ## Removing Manager Redundancy 1. Go to **Administration > Redundancy > Cluster Configuration**. ![Remove Cluster Configuration](./images/redundancy_11.png) 2. Click **Remove Cluster Configuration**, then confirm - If the Application URL was set to the proxy IP or domain, no changes are sent to the connected appliances - The system enters maintenance mode for a few minutes - Once complete, you will have two independent Spectra Detect Managers. Review [Post De-clustering Actions](#post-de-clustering-actions) after this step to retire the unused manager. ### If the Secondary is unavailable during de-clustering - You can still remove the cluster following the directions in Step 2 above. If the offline secondary manager was permanently removed or destroyed, no additional cleanup is required after Step 2. - If you have access to the server, you can manually run the below command to clean up the de-clustered Spectra Detect Manager that was offline during the initial de-cluster action. ```bash sudo /bin/cluster_manage destroy ``` If the secondary comes back online after de-clustering, it must be manually de-clustered. Contact [ReversingLabs Support](mailto:support@reversinglabs.com) for help. ## Post De-clustering Actions **Important: **After de-clustering, shut down or remove one of the Spectra Detect Managers.**** All connected appliances continue communicating through the proxy or load balancer, which remains configured to route to the active manager. Remove the inactive or retired manager from the proxy or load balancer configuration to prevent connection attempts to a non-existent node. If you plan to re-establish redundancy with a new secondary manager, repeat the steps in [Creating Manager Redundancy](#creating-manager-redundancy). --- ## Spectra Detect Token Management — API Authentication and Access Control **Important: This section is available in OVA and Kubernetes microservices deployments.** ***Administration > Token Management*** Under **Token Management**, you can manage authentication tokens, which are per-user keys for authenticating to the Spectra Detect Manager APIs. ## Adding tokens To create a new token, click **Add Token** and, from the drop-down list, select a **User**. Clicking **Save**, **Save and add another**, or **Save and continue editing** assigns a token to the selected user. Optionally, do the following: | Icon | Description | |----------------------|----------------------------------------------------------------------------------------------------------| | Change selected user | Opens a new window where you can change the selected user's information. | | Add another user | Opens a new window where you can add another user. | | View selected user | Leaves the **Token Management** page and opens the selected user's information in the same tab. | ## Searching for tokens The **Token Management** page contains a search bar for finding specific tokens. The search is case-insensitive. To search for a token, enter an exact or partial key or username, and then click **Search**. To view the full list of tokens again, click the link denoting the total number of tokens in the parentheses next to the number of search results. The token list supports multi-column ordering. The contents of every column can be sorted in ascending or descending order by clicking the column name. When another column name is clicked, the ordering priority is shifted to the new column and indicated by numbers next to the column names. ## Editing tokens To edit an existing token, from the list on the **Token Management** page, click its key. From this page, you can update any information entered while [adding the token](#adding-tokens). This page also contains a **History** link that displays the history of changes related to that particular token. ## Deleting tokens To delete one or more tokens, from the **Token Management** page, find and select the tokens you want to delete. Then, under **Action**, select **Delete Selected Tokens** and click **Go**. Alternatively, find and click the key of the token you want to delete, and at the bottom of the page, click **Delete**. On the next page, confirm the deletion of this token. --- ## Updating Spectra Detect — .bin Upload, Spectra Intelligence, and Air-Gapped Upgrades **Important: This section is only available in OVA deployments.** This chapter describes update management using the graphical user interface. To update appliances using the Manager APIs, refer to the [Manager API Guide](../API/ManagementAPI.md). ## Managing Update Packages The Appliance Management page on the dashboard is the central place for updating the Manager and any connected appliances. To start updating, visit the **Administration ‣ Update Bin Management** page to manage update packages in one of two ways: 1. Manually, by uploading the update package. When an update is available, ReversingLabs will provide a .bin file via email or other file transfer method. Click **Upload .bin** to open the file upload dialog and find either the .bin file or an archive containing multiple .bin files on the local filesystem. In the case when the Manager instance is not connected to Spectra Intelligence, only the upgrade images manually uploaded to the Manager will be visible. The System Update page works in air-gapped situations. 2. By retrieving the update from Spectra Intelligence. The prerequisite is that the Manager is connected to the Spectra Intelligence cloud (configurable in **Administration ‣ Spectra Detect Manager ‣ Spectra Intelligence**). When a new update is available in Spectra Intelligence, the "System is up to date" message on the page is replaced by a button allowing the user to download the update package. Update .bin files from Spectra Intelligence are visible only if their version is newer than that of connected appliances. If multiple downloads are happening at the same time, these files are downloaded one by one. Under the **Upload .bin** button, there is an indicator of the total and used disk space in the Manager update repository. The update repository disk space is set to 50 GB by default. To increase the repository size, contact [ReversingLabs Support](mailto:support@reversinglabs.com). The rest of the page is a table showing all the .bin files that were either uploaded to the system, or can be downloaded in case the Manager is connected to Spectra Intelligence. The table consists of the following fields: - **Appliance Type**, showing the appliances connected to the Manager instance - **Version**, showing the current version of each appliance - **Size**, showing the size of the .bin file - **Version Prerequisite**, showing the version of the appliance that is required for the update - **Action**, where the user can choose to either **Download** the .bin file for a particular appliance or **Delete** the existing file The data contained here can be filtered by the appliance type. If another .bin file of the same version is uploaded, the old one is overwritten. Upon migration, the existing .bin files will remain on the system and will remain visible. ## Updating Appliances When an update is available for the Manager or a connected appliance, it is indicated by a **Update** button in the Updates column on the Appliance Management page on the dashboard. Every appliance can be updated independently of other appliances of the same type. Appliances connected using the `http://` protocol will display the following message: "For security reasons, appliances cannot be updated or sync YARA rules without HTTPS URLs. Change the appliance URL to HTTPS to update this appliance or synchronize YARA rules." Clicking the button opens the confirmation dialog where the update process can be initiated (by clicking **Update**) or canceled (by clicking **Cancel**). An update progress bar will replace the **Update** button for the appliance that is being updated. If the appliance update is successful, an "Updated to [version]" notification is displayed in the Updates column for the updated appliance. The appliance web server and services will be restarted. Note that in some cases the whole appliance may reboot. Because of the web server restart, it is recommended to notify all appliance users of the impending update before it is applied. If the update process fails, the *Update* button changes to **Retry**, with the additional option to **Download Logs** for more information on why the update failed. The dialog allows the user to try updating the appliance again (by clicking **Retry**), as well as to view a detailed log and inspect why the update failed (by clicking **Log**). ### Appliance Update Policies **Sequential Upgrade Policy** Spectra Detect and Spectra Analyze appliances can be directly upgraded from any minor version to any minor version of the next major version. Major/feature releases are indicated by the first two numbers of the version number, while the third number is reserved for minor/patch releases. For example, users who want to upgrade their Spectra Analyze 5.9.2 to Spectra Analyze 5.10.3 can do that without having to manually install the intermediate 5.9.3 or 5.10.1 versions. Note that major versions are not skipped during upgrades, so upgrading directly from 5.8.X to 5.10.X is not possible without installing 5.9.X. --- ## Spectra Detect User Management — Accounts, Access Control, and LDAP **Important: This section is available in OVA and Kubernetes microservices deployments.** ***Administration > User Management*** ## Adding users To create a new user, click **Add User** and enter the following obligatory information: - **Username** - **Password** - **Password Confirmation** - **Email Address** Optionally, under **Personal Info**, provide the user's **First Name** and **Last Name**. To save your changes, click **Save**, **Save and add another**, or **Save and continue editing**. ## Searching for users The **User Management** page contains a search bar for finding specific users. The search is case-insensitive. To search for a user, enter an exact or partial username or email address, and then click **Search**. To view the full list of users again, click the link denoting the total number of users in the parentheses next to the number of search results. The user list supports multi-column ordering. The contents of every column can be sorted in ascending or descending order by clicking the column name. When another column name is clicked, the ordering priority is shifted to the new column and indicated by numbers next to the column names. You can also use the filters in the right sidebar to filter the list of users by last login, date joined, and status (active, superuser). ## Editing users To edit an existing user's settings, from the list on the **User Management** page, click their username. From this page, you can update any information entered while [adding the user](#adding-users). You can also update the following **Permissions**: - **Active**: designates whether this user should be treated as active. It is recommended you deactivate users by clearing this option, instead of deleting accounts. - **Superuser**: designates that this user has full access to all features. The **Important dates** section lists the dates when this user was added and when they last logged in. ## Removing users You can remove users in the following ways: - **Deleting users**: from the **User Management** page, find and select the users you want to delete. Then, under **Action**, select **Delete Selected Users** and click **Go**. Alternatively, find and click the username of the user you want to delete, and at the bottom of the page, click **Delete**. On the next page, confirm the deletion of this user. - **Deactivating users**: from the **User Management** page, find and click the username of the user you want to deactivate. Under **Permissions**, clear the **Active** checkbox, and then click **Save**. **Tip: Best practice** It is not recommended you delete users. You should instead deactivate them. An administrator cannot remove or deactivate their own account while they are logged in. An administrator account can only be removed or deactivated by another administrator account. ## User directory settings Spectra Detect Manager supports user account management with the Lightweight Directory Access Protocol (LDAP). LDAP authentication can be enabled and configured under [Administration > Authentication](../Admin/Authentication.md). Importantly, the existing local accounts are not managed or in any way affected by LDAP when its configuration changes. Similarly, any username conflicts between LDAP and existing user accounts resolve in favor of the existing account. For example, if an account named "admin" exists on the appliance, it is not possible to assign the same name to an LDAP account. --- ## Spectra Detect YARA Sync — Ruleset Synchronization Across Appliances **Important: This section is only available in OVA deployments.** The YARA Sync page (**Administration ‣ YARA Synchronization**) allows users to easily track the status of YARA ruleset synchronization between connected appliances, and trigger a manual synchronization if rules are not up-to-date. The Manager stores all synchronized rules in a local database and becomes the single source of truth for all connected appliances. When *YARA ruleset synchronization* is enabled, the YARA Sync page displays a table of all appliances connected to the Manager and their YARA ruleset synchronization status. Any connected [Spectra Analyze](/SpectraAnalyze/) appliances must have YARA Synchronization enabled (**Administration ‣ Spectra Detect Manager ‣ General ‣ Synchronization**) to properly display the current status and synchronize rulesets. Appliances can show one of the following statuses: - InSync - OutOfSync - Error - Unavailable - PendingNew - Disabled - NoRules Workers poll the Manager for rule changes every minute. Spectra Analyze appliances push new rules to the Manager as soon as they are created, and pull new rules every 5 minutes. Appliances that are *Not In Sync* can be manually synchronized at any time by clicking the *Start YARA Sync* button in the far right column of the table. Rulesets created on Spectra Analyze appliances before YARA synchronization was enabled will not synchronize to the Manager until the user changes their status or modifies them in any way. Rules present on the Manager, however, will synchronize to newly connected Spectra Analyze appliances regardless of when they were created. Apart from new rulesets, changes in existing rulesets will be synchronized as well. If a ruleset is disabled or deleted on one appliance, its status will be distributed to other appliances. In case of Workers, disabled rulesets will be removed until re-enabled on another appliance. When enabled again, rulesets will be synchronized on the Worker as if they have been newly created. Since all rulesets have owners, their user accounts will be mirrored to other connected appliances, but won’t be able to log into that instance until an administrator enables their account by assigning it a password. YARA Ruleset Restrictions - Naming restrictions: - YARA ruleset names must be between 3 and 48 characters. - The underscore ( _ ) should be used instead of spaces, and any other special characters should be avoided. Ruleset names should only use numbers (0-9) and a-z/A-Z letters. - Ruleset size restrictions: - A ruleset file should not be larger than 4 MB. - A ruleset file should not contain more than 5000 individual rules. - A ruleset larger than 1 MB (1048576 bytes) cannot be saved and run in the [Spectra Intelligence](/SpectraIntelligence/) cloud. - File size restrictions: - YARA rulesets on Spectra Analyze are not applied to files larger than 700 MB. Only rules that have been successfully compiled on Spectra Analyze can be synchronized. --- ## Spectra Detect Administration — Manager Settings, Users, and Licensing --- ## Spectra Detect Management API — SDM Authorization, Backup, and Appliance Setup **Tip: This is a companion guide for the Manager's API endpoints. For a full reference, including the OpenAPI specification, see the Manager API reference (available in the help menu on the Manager appliance).** Here are the steps covered in this guide: 1. [Obtain an authorization token](#api-token) 2. [Back up and restore the Manager](#back-up-and-restore-the-manager) 3. [Set up the Manager](#set-up-the-manager) 4. [Connect appliances to the Manager](#connect-appliances-to-the-manager) 5. [Configure appliances](#configure-appliances) 6. [Configure connectors](#configure-connectors) 7. [Configure central logging](#configure-central-logging) **Tip: For setups using a self-signed certificate, add `-k` or `--insecure` before the appliance URL in the curl examples.** ## Obtain an authorization token {#api-token} To use the Manager API, an authorization token needs to be included in all API calls. You can obtain this token using the authentication API or the web interface. The authorization token is a string of alphanumeric characters, between 30 and 60 characters in length. The authorization token should be provided in the HTTP `Authorization` header. The value of this field must be in this format: ``` Token exampletoken ``` where `exampletoken` should be replaced with the user's own authorization token string. Call the authentication API with your username and password using the `multipart/form-data` content type. ``` POST /api/v1/auth/token/ ``` Form parameters: - `username` - `password` Request example: ```bash curl -X POST 'https://c1000.example.com/api/v1/auth/token/' --form "username=user123" --form "password=pass123" ``` Response example: ```json { "token": "" } ``` This token has to be included in all further requests to the API in the *Authorization* header. For example: ```bash curl -H "Authorization: Token " https://c1000.example.com/api/configuration/appliances ``` ## Back up and restore the Manager In certain cases, you may want to perform a complete backup of the Manager before performing any other actions. Backup files contain [central configuration settings](../Config/ApplianceConfiguration.md), user accounts on the Manager instance, a database of connected appliances and all of their configuration files, appliance groups, and synchronized YARA rules. Uploading the backup archive to a clean machine restores the data stored in the Manager database, but configuration settings first need to be applied to configuration groups to take effect. The steps are: 1. [Create a backup file](#create-a-backup-file) 2. [Find out the backup status](#find-out-the-backup-status) 3. [Download backup file](#download-backup-file) 4. [Upload and restore the Manager from backup](#upload-and-restore-the-manager-from-backup) ### Create a backup file ``` POST /api/v1/backup/create/ ``` Creates a backup file that can be downloaded with the [`GET /api/v1/backup/download/`](#download-backup-file) endpoint. ### Find out the backup status ``` GET /api/v1/backup/ ``` Returns the current backup status and availability. Response: ```json5 { "status": "error", "message": "idle", // when the message is 'idle', the backup is ready "backup_filename": "example" } ``` ### Download backup file ``` GET /api/v1/backup/download/ ``` Downloads the backup file. ### Upload and restore the Manager from backup ``` POST /api/v1/backup/restore-upload/ ``` Uploads the backup file to the Manager. Form parameters: - `file`: the backup file ## Set up the Manager Perform the following actions to set up the Manager. This guide is a reference for an enterprise-ready deployment of Spectra Detect, with redundancy and password rotation requirements, for example. Skip the steps that are not relevant to your desired setup. If you have additional requirements that are not covered by this guide, you can always find the complete API reference on the Manager itself (Help menu). The steps for an initial setup are: 1. Set up authentication: 1. (Get authorization token - already covered in [the previous section](#api-token)) 2. [Change admin password](#change-admin-password) 2. [Configure network settings](#configure-network-settings): 1. Upload SSL certificate 2. Configure the application URL 3. List the allowed hosts 4. Set network time sync 3. [Configure user authentication](#configure-user-authentication) 1. Federated login 2. SSH access 4. [Configure Spectra Intelligence](#configure-spectra-intelligence) 5. [Update the Manager](#update-the-manager) 6. [License the Manager](#license-the-manager) 7. [Set up logging and alerting](#set-up-logging-and-alerting) 1. SNMP 2. SMTP 3. System alerting 4. Central logging 8. [Enable synchronization and redundancy](#enable-synchronization-and-redundancy) 9. [Set a timezone](#set-a-timezone) ### Change admin password First, you need to get the ID of the admin user. It's almost certainly `1`, but you can get all users and their IDs by calling this endpoint: ``` GET /api/v1/authentication/users/ ``` Returns all users on the appliance. Response example: ```json { "count": 1, "next": null, "previous": null, "results": [ { "id": 1, "username": "admin", "email": "hello@example.com", "first_name": "", "last_name": "", "directory_name": "", "directory_type": 0, "last_login": "2024-02-07T12:10:04.744401Z", "date_joined": "2024-01-03T03:01:30.166022Z", "is_superuser": true, "is_active": true } ] } ``` Then change the administrator password: ``` PUT /api/v1/authentication/users/{id}/password/ ``` Path parameters: - `id`: user ID Request body example: ```json5 { "password": "Correct Horse Battery Staple" } ``` ### Add or remove users #### Add a user ``` POST /api/v1/authentication/users/add/ ``` ##### Request body ```json5 { "username": "string", "email": "user@example.com", "first_name": "string", "last_name": "string", "password": "string" } ``` ##### Response example ```json { "id": 0, "username": "string", "email": "user@example.com", "first_name": "string", "last_name": "string", "directory_name": "string", "directory_type": 0, "last_login": "2019-08-24T14:15:22Z", "date_joined": "2019-08-24T14:15:22Z", "is_superuser": true, "is_active": true } ``` #### Remove a user ``` DELETE /api/v1/authentication/users/{id}/delete/ ``` ##### Path parameters - `id`: user ID #### Examples Example adding and deleting a user: ```bash curl --url https://{server}/api/v1/authentication/users/add/ --header 'Authorization: Token ' --json '{"username":"hello","email":"example@example.com","password":"example"}' curl --url https://{server}/api/v1/authentication/users/3/delete --header 'Authorization: Token ' --request DELETE ``` ### Configure network settings The Manager can operate without a valid SSL certificate, but to maximize security in communication between users and the appliance, you may want to first upload a valid SSL certificate to the Manager. #### Upload a certificate To upload the certificate to the Manager, use the Upload Certificate endpoint. ``` POST /api/v1/system/upload-certificate/ ``` ##### Form parameters - `file`: combined PEM file containing both private key and certificate (in standard PEM format) The file should contain both the private key and certificate in a single PEM file. You can create this combined file using: ```bash cat private.key certificate.pem > combined.pem ``` ##### Response codes - `200`: certificate uploaded successfully - `400`: validation error - `500`: server error `curl` code sample: ```bash # Add --insecure before the URL if you are using a self-signed SSL certificate curl -X POST https://{server}/api/v1/system/upload-certificate/ -H "Authorization: Token " -F "file=@combined.pem" ``` While the SSL certificate upload is optional, you *must* set up the base URL and allowed hosts. This request also allows completing your SSL setup. #### Configure the network ``` POST /api/v1/config/network/ ``` ##### Request body ```json5 { "base_url": "string", "allowed_hosts": "string", // allowlisted IP addresses or URLs, separated by a newline (`\n`) character "certificate_path_displayname": "string", "certificate_key_path_displayname": "string", "reset_ssl": true } ``` ###### Request body example ```json { "base_url": "c1000.example.com", "allowed_hosts": "worker1.example.com\nworker2.example.com", "certificate_path_displayname": "/my/default/path", "certificate_key_path_displayname": "/my/default/path", "reset_ssl": true } ``` #### Enable network time synchronization ``` POST /api/v1/config/ntp/ ``` ##### Request body ```json5 { "enable": true, "servers": "string", // must be separated by a newline (`\n`) character "timezone": "string" } ``` ### Configure user authentication After configuring the minimum necessary settings for proper networking of the Manager, continue with provisioning user access. In enterprise deployments, a federated (single sign-on) login is often used. This guide references only some of the available fields - for a complete overview, check the API reference on the Manager itself. ``` POST /api/v1/config/directory/ ``` Sends LDAP/OIDC configuration to the Manager. After sending the configuration with `POST /api/v1/config/directory/`, you can verify that the configuration is updated using `GET /api/v1/config/directory/`. ### Enable SSH root login **Warning: Dangerous! Some services like password rotation solutions require root SSH access to the devices they are managing. This opens up the attack surface considerably.** ``` POST /api/v1/system/config/sshd/ ``` Sends the SSH configuration to the appliance. ##### Request body ```json5 { "banner_content": "string", "macs": "string", "ciphers": "string", "kexalgorithms": "string", "authorized_keys_file": "string", "client_alive_interval": "string", "client_alive_count_max": "string", "permit_root_login": "yes" } ``` ### Configure Spectra Intelligence Next, configure the connection between Spectra Intelligence and the Manager. This is a prerequisite for licensing the Manager using Spectra Intelligence. In addition, connecting it to Spectra Intelligence allows it to download upgrade binaries. ``` POST /api/v1/config/ticloud/ ``` Sends the configuration for Spectra Intelligence to the Manager. ##### Request body example ```json5 { "enable": true, "url": "https://appliance-api.reversinglabs.com", "user": "u/myusername", "token": "exampleABC123", // your password "timeout": 10 // in seconds } ``` To verify the changes, send a GET request to the same endpoint: ``` GET /api/v1/config/ticloud/ ``` ### Update the Manager Get the list of available upgrades: ``` GET /api/v1/update-system/available-update/ ``` #### Response example ```json5 { "filename": "string", "sha1": "string", "version": "string", // version introduced by the update "requires": "string", // required version to use this update file "time": "2019-08-24T14:15:22Z" // build time } ``` Then update the Manager to the latest available version: ``` POST /api/v1/update/latest/ ``` #### Examples ```bash curl -X GET https://{server}/api/v1/update-system/available-update/ -H 'Authorization: Token ' curl -X POST https://{server}/api/v1/update/latest/ -H 'Authorization: Token ' ``` ### License the Manager The Manager can be licensed in two ways: 1. [Using Spectra Intelligence](#using-spectra-intelligence) 2. [Using a license file](#using-a-license-file) #### Using Spectra Intelligence To configure Spectra Intelligence on the Manager, use the following endpoint. ``` POST /api/v1/license/configure/cloud/ ``` Performs a licensing request using Spectra Intelligence. ##### Request body ```json5 { "url": "https://appliance-api.reversinglabs.com", "user": "string", "token": "string", "timeout": 1, "proxy_enabled": true, "proxy_host": "string", "proxy_port": 0, "proxy_user": "string", "proxy_password": "string" } ``` ##### Example ```bash curl -X POST 'https://c1000.example.com/api/v1/license/configure/cloud' --header 'Content-Type: application/json' --header 'Authorization: Token exampletoken' --data '{ "url": "https://appliance-api.reversinglabs.com", "user": "string", "token": "string", "timeout": 1, "proxy_enabled": true, "proxy_host": "string", "proxy_port": 0, "proxy_user": "string", "proxy_password": "string" }' ``` Conversely, perform a GET request to retrieve the configuration: ``` GET /api/v1/license/configure/cloud/ ``` #### Using a license file The alternative to Spectra Intelligence is manually uploading a license file. To do so, first create a unique machine ID: ``` POST /api/v1/license/machine-id/generate/ ``` ##### Example ```bash curl -X POST https://c1000.example.com/api/v1/license/machine-id/generate/ -H 'Authorization: Token exampletoken' ``` When the Manager responds with the machine ID, email it to [ReversingLabs Support](mailto:support@reversinglabs.com). We will respond with a license file that can be uploaded to the Upload License File endpoint: ``` PUT /api/v1/license/upload/ ``` ##### Form parameters - `file`: license file ##### Example ```bash curl -X PUT https://example.c1000.com/api/v1/license/upload -H 'Authorization: Token exampletoken' -F 'file=@example_license_file' ``` ### Set up logging and alerting This chapter covers methods to monitor performance of the Manager itself. All the endpoints listed below also accept the GET method, which returns the current configuration. #### SNMP Send SNMP configuration to the Manager. The Manager uses SNMP v2c. ``` POST /api/v1/config/snmp/ ``` ##### Request body ```json5 { "enable": true, "community": "string", "trap_sink_enable": true, // enable sending traps "trap_community": "string", // community string for traps "trap_sink": "string", // trap receiver address // 👇 The Manager sends a trap when this value is exceeded // (percentage of used CPU threads over 1, 5 and 15 minutes) "trap_avg_load_threshold_1min": 1, "trap_avg_load_threshold_5min": 1, "trap_avg_load_threshold_15min": 1, "trap_memory_threshold": 1, // percentage of used memory "trap_disk_threshold": 1 // percentage of used disk } ``` #### SMTP ``` POST /api/v1/config/smtp/ ``` Sends SMTP configuration to the Manager. ##### Request body ```json5 { "host": "string", // SMTP relay server "port": 65535, "user": "", "password": "", "password_set": false, "use_tls": false, "default_from_email": "user@example.com" } ``` #### System alerting Send configuration for syslog usage to the Manager. ``` POST /api/v1/config/systemalerting/ ``` ##### Request body ```json5 { "syslog_enable": true, "syslog_host": "string", "syslog_port": 65535, "syslog_transport_protocol": "TCP", "syslog_enable_audit_logs": true } ``` #### Central logging Configure central logging on the Manager. ``` POST /api/v1/config/central_logging/ ``` ##### Request body ```json5 { "central_logging__enable": true, "central_logging__retention_period": 1, // in days "central_file_storage__enable": true, "central_file_storage__ttl": 1, // how long to keep files on the Manager (in hours) "central_file_storage__file_size_limit": 1, // in MiB "central_storage_general__min_free_space": 10, // in GiB } ``` ### Enable synchronization and redundancy One last step before completing the setup of the Manager is to enable the synchronization of YARA rulesets, as well as setting up a redundant Manager that takes over in case of a failure of the primary. #### YARA sync ``` POST /api/v1/config/sync/ ``` Enables or disables the synchronization of YARA rulesets across connected appliances. ##### Request body ```json5 { "yararulesets_enable": true } ``` #### Set up a redundant cluster **Note: The prerequisite for this is to have another Manager already operational and accessible on the network.** The first step is to create a cluster and set up the basic configuration: ``` POST /api/v1/cluster/create/ ``` ##### Request body ```json5 { "local_ip": "192.168.0.1", // IP address of the primary Manager "remote_ip": "192.168.0.1", // IP address of the secondary Manager "vpn_network": "string", // virtual private network range in CIDR notation; subnet mask can only be `/29` or lower "virtual_ip": "192.168.0.1", // shared IP address to access the Manager when in redundant mode "failover_timeout": 3 // seconds between the unavailability of the primary and activation of the redundant Manager } ``` After this, you must perform a mutual authentication between the two appliances: ``` POST /api/v1/cluster/exchange_keys/ ``` ##### Request body ```json5 { "values": { "local_ip": "string", // primary Manager IP address "remote_ip": "string", // secondary Manager IP address "remote_url": "string", // secondary Manager URL "user": "string", // admin "pwd": "string" // admin password } } ``` Finally, verify that redundancy works: ``` POST /api/v1/cluster/verify/ ``` Performs checks between appliances. ##### Request body ```json5 { "remote_ip": "192.168.0.1", // IP address of the secondary Manager "section": "ssh" // aspect of redundancy to check (can be `ssh` or `versions`) } ``` ##### Response example ```json5 { "return_code": 0, // HTTP response status code "message": "string", // request-related message "messages": [ "string" // messages related to redundancy ], "reduser_ssh_connect": true, // is SSH connectivity established? "postgres_ssh_connect": true, // is the redundant database accessible? "rld_ssh_tunnel_connect": true, // can an SSH tunnel be established? "version_local": 0, // version of the primary Manager "version_remote": 0 // version of the secondary Manager } ``` ### Set a timezone This section covers listing and setting the timezone of the Spectra Detect Manager. #### List timezones List available timezones for the Spectra Detect Manager. The response contains a list of timezones that can be set. ``` GET /api/v1/system/config/list-available-timezones/ ``` ##### Response example ```json5 { "timezones": [ "string" ] } ``` #### Set the timezone of Spectra Detect Manager ``` POST /api/v1/system/config/timezone/ ``` ##### Request body ```json5 { "timezone": "string" } ``` ## Connect appliances to the Manager This section covers the following actions: 1. [Add appliances to the Manager](#add-appliances-to-the-manager) 2. [Authorize appliances](#authorize-appliances) 3. [Update appliances](#update-appliances) 4. [Upload SSL certificate to appliances](#upload-ssl-certificate-to-appliances) 5. [Check connected appliances](#check-connected-appliances) ### Add appliances to the Manager **Note: Appliances first must be up and running.** To configure appliances with the Manager, start with adding them: ``` POST /api/v1/appliances/create/ ``` #### Request body ```json5 { "name": "string", "type": "a1000", // possible values: a1000, tiscale-worker, tiscale-hub "url": "http://example.com", "snmp_community": "string", "haproxy_password": "string" // load balancer password (Hub) } ``` The appliance `type` accepts legacy names for the products: - `a1000` is Spectra Analyze - `tiscale-worker` is Spectra Detect Worker - `tiscale-hub` is Spectra Detect Hub The response object format is the same as the request object. ### Authorize appliances After being added to the Manager, appliances must be authorized. The authorization process differs, depending on the appliance. The API endpoint is the same, but two different methods must be used: **GET** for Hubs and Workers, and **PUT** for Spectra Analyze. **Tip: Use `GET /api/v1/configuration/appliances/` to get the IDs of all connected appliances.** #### Hub, Worker ``` GET /api/v1/appliances/{id}/authorization/ ``` ##### Path parameters - `id`: ID of the appliance #### Spectra Analyze ``` PUT /api/v1/appliances/{id}/authorization/ ``` ##### Path parameters - `id`: ID of the appliance ##### Request body ```json5 { "token": "string" } ``` ### Update appliances Get the list of available upgrades: ``` GET /api/v1/package-management/ ``` Returns the list of available update binaries for connected appliances. ##### Response body ```json5 { "bins": [ { "type": "string", "version": "string" } ], "used_space": 0, // how much space is used up on the appliance "allowed_space": 0, // how much space is left on the appliance "is_ticloud_data_available": true // is Spectra Intelligence data on upgrade binaries available } ``` After retrieving the list of available updates, use the appliance type and version from the `bins` object to download one of the available updates: ``` POST /api/v1/package-management/{appliance_type}/{version}/ ``` Downloads an update package from the Spectra Intelligence cloud. Path parameters: - `appliance_type`: what type of appliance? - `a1000` stands for Spectra Analyze - `c1000` stands for Spectra Detect Manager - `tiscale-hub` stands for Spectra Detect Hub - `tiscale-worker` stands for Spectra Detect Worker - `version`: version of the appliance Once the update downloads, the following request will upgrade a single appliance (upgrading multiple appliances at the same time isn’t supported). To upgrade an appliance, find its name using `GET /api/v1/configuration/appliances/` and include it in the request body (see example below). ``` POST /api/upgrade/start/ ``` Starts appliance upgrade. Request body: ```json5 { "appliance_name": "string" } ``` You can check the upgrade status at any point: ``` POST /api/upgrade/status/ ``` Request body: ```json5 { "appliance_name": "string" } ``` Response: The body displays the current status, which can be: - `idle`: there is no update happening - `pending`: an update is planned - `downloading`: an update is currently being downloaded to the appliance - `installing`: an update is installing - `error`: there was an error installing an update The body will also contain the entire upgrade log, if available. If not, it will contain the message "Update log not available." Here's a list of `curl` commands corresponding to the steps above: ```bash curl -X GET https://{server}/api/v1/package-management/ -H 'Authorization: Token ' curl -X POST "https://{server}/api/v1/package-management/tiscale-worker/3.4.0/" -H "Content-Type: application/json" -H 'Authorization: Token ' -d '{}' curl -X POST https://{server}/api/upgrade/start/ -H 'Authorization: Token ' -d '{"appliance_name":""}' curl -X POST https://{server}/api/upgrade/status/ -H 'Authorization: Token ' -d '{"appliance_name":""}' ``` ### Upload SSL certificate to appliances **Note: Hub and Worker only. Uploading SSL certificates to Spectra Analyze through the Manager is not possible.** ``` POST /api/v1/appliances/upload-certificate/{id}/ ``` Uploads an SSL certificate to an appliance. Path parameters: - `id` - ID of the appliance Form parameters: - `file` - certificate file The certificate file must be a combined file that includes both the private key and the certificate. You can generate it with a command such as: ``` cat private.key certificate.pem > combined.pem ``` ### Check connected appliances After adding appliances, verify their connectivity: ``` GET /api/v1/configuration/appliances/ ``` This endpoint returns a list of appliances connected to the Manager. In the response, the structure of the fields in the `results` array is as follows: ```json5 { "id": 0, "appliance_name": "string", "type": "a1000", "release": "string", "cluster": "string", // "standalone" if not in a group "state": "string", // whether primary or secondary "appliance_hostname": "string" } ``` ...and licensing status: ``` GET /api/v1/appliances/licensing/ ``` Returns the licensing status of all appliances connected to the Manager. ## Configure appliances This section describes how to centrally configure and manage connected appliances. ### Licensing connected appliances There are two ways to license connected appliances: using Spectra Intelligence and using license files. #### Using Spectra Intelligence To license connected appliances using the Spectra Intelligence cloud, configure it for that appliance, either through a group ([Standard configuration groups](#standard-configuration-groups), [Hub groups](#hub-groups)) or individually ([Individual configuration: Spectra Analyze](#individual-configuration-spectra-analyze)). The relevant portion required in the body of the request is: ```json5 { "content": { "cloud__enabled": true, "cloud__url": "data.reversinglabs.com", "cloud__user": "example_username", "cloud__token": "example_password", "cloud__token_set": true } } ``` #### Using license files To license connected appliances using license files, they all need to have a machine ID. Machine IDs can be generated for appliances without one, or generated for all appliances at once. If regenerating machine IDs for all appliances, those that already have a license will have to be licensed again. ``` GET /api/v1/license/ ``` Generates a machine ID only for those appliances that don't have it. The response contains individual objects (each of which represents an appliance) with the following fields: ```json5 { "status": "VALID", "product": "TISCALE_WORKER", "machineId": "string", "version": 0, "serialNumber": "148d7a56-176d-413b-9eb5-c62f6f5bb67a", "licenseType": "WILDCARD", "company": "string", "expiryDate": "2019-08-24", "creationDate": "2019-08-24" } ``` If, for some reason, **all** machine IDs have to be generated, use `GET api/v1/appliances/licensing/machine-id/`. Send one or more machine IDs to ReversingLabs support and, once we respond with license files, upload them to the Manager using the `PUT /api/v1/appliances/licensing/` endpoint. The Manager will automatically match the license to the appropriate appliance. ``` PUT /api/v1/appliances/licensing/ ``` Uploads a license file for one of the connected appliances. Form parameters: - `file` - license file to be uploaded ### Standard configuration groups This section describes actions related to creating and managing configuration groups for Spectra Detect Workers. Start with creating a configuration group: ``` POST /api/v1/config-group/create/ ``` Creates a Worker configuration group. ##### Request body ```json5 { "name": "string" } ``` Then, add the appliances and specify their configuration. Both PUT and PATCH methods are supported. **Note: In this guide, we display only some of the possible configuration options as there are almost 400 configurable fields. See the full API reference on the Manager for all options.** ``` PATCH /api/v1/config-group/content/{group_name}/ ``` Saves configuration (but doesn't apply it). The body must be a JSON object with two top-level fields: `appliances` and `content`. Path parameters: - `group_name`: name of the configuration group Request body: ```json5 { "content": { "snmp__enable": true, "snmp__community": "^$", "snmp__trap_sink_enable": true, "snmp__trap_community": "^$" // ... ``` Check that the configuration is correct: ``` GET /api/v1/config-group/retrieve/{group_name}/ ``` Returns the configuration for the group. Path parameters: - `group_name` - name of the configuration group Apply the configuration: ``` PUT /api/v1/config-group/update/{group_name}/ ``` Applies the configuration set in the previous step. Path parameters: - `group_name`: name of the configuration group Request body: ```json5 { "applyconfiguration": [ "string" // list of appliance names indicating which appliances in the group will receive group configuration ] } ``` - `applyconfiguration`: a list of strings where each string is the name of an appliance in the group. Group appliance configuration will be applied to the appliances listed. To verify that the configuration is applied, call the following endpoint: ``` GET /api/v1/appliances/ ``` Get a list of appliances with their configuration details. The response contains an array called `results`, with objects inside. Each object corresponds to one connected appliance. ```json5 { "id": 0, "uuid": "095be615-a8ad-4c33-8e9c-c7612fbf6c9f", "ip": "string", "status": "string", "authorized": true, "online": true, "latency": 0.1, "maintenance": true, "last_seen_online": "2019-08-24T14:15:22Z", "system_data": {}, "haproxy_password": "string", "created": "2019-08-24T14:15:22Z", "updated": "2019-08-24T14:15:22Z", "group_name": "string", "redundancy_status": "string", "configured_redundancy_status": "string", "redundant_with": "string", "connected_workers": [], "application_url": "string", "available_update_version": "string", "has_update_log_endpoint": true, "last_applied_config_timestamp": "2019-08-24T14:15:22Z", "configuration_status": "changes_awaiting_apply", // check this field "configuration_status_details": {} } ``` If you have applied the configuration to all the appliances in a configuration group, then find an object that contains the `group_name` corresponding to your configuration group, and inspect its `configuration_status`. The possible values are: - `changes_awaiting_apply` - `pending` - `error` - `done` - `mismatch` - `unsupported` - `no_configuration_group` If you have several groups with different configurations, use this query to list them: ``` GET /api/v1/config-group/list/ ``` Lists configuration groups. Query parameters: - `page`: response page (if the contents are too large for a single response) - `page_size` - number of results to return per page If necessary, remove a group: ``` DELETE /api/v1/config-group/delete/{group_name}/ ``` Removes a group. Path parameters: - `group_name`: name of the group to remove Here's a list of `curl` commands corresponding to the steps above: ```bash # Add --insecure before the URL if you are using a self-signed SSL certificate curl -X POST https://{server}/api/v1/config-group/create/ -H 'Authorization: Token ' -H 'Content-Type: application/json' -d '{"name":"TestGroup1"}' curl -X PATCH https://{server}/api/v1/config-group/save/TestGroup1/ -H 'Authorization: Token ' -H 'Content-Type: application/json' -d '{"appliances":["Test"]}' curl -X PATCH https://{server}/api/v1/config-group/save/TestGroup1/ -H 'Authorization: Token ' -H 'Content-Type: application/json' -d '{"content": { "cloud__enabled":true, "cloud__url":"https://appliance-api.reversinglabs.com", "cloud__user":"Test", "cloud__token":"Test", "cloud__timeout":60, "cloud__proxy_enabled":false, "cloud__proxy_host":"", "cloud__proxy_port":null, "cloud__proxy_user":"", "cloud__proxy_password":"", "cloud__max_fetch_file_size":500 } }' curl -X PATCH https://{server}/api/v1/config-group/update/TestGroup1/ -H 'Authorization: Token ' -H 'Content-Type: application/json' -d '{"applyconfiguration":["Test"]}' ``` To reset the configuration of a group to default and apply the current group settings, use the following query: ``` POST /api/v1/config-group/resolve-mismatch/group_name/ ``` Resolves a mismatch in the configuration of a group by resetting it to default and applying the current group settings. Returns a `204` status code. ### Hub groups A special case of configuration groups is using **Hub groups**. These are just like regular configuration groups, but you can also create and manage Hubs, including all the functionalities that they offer. The prerequisite for the following actions is adding at least one Hub using `POST /api/v1/appliances/create/` (you can't have a Hub group without a Hub). The first step is to create a Hub group and add **all Hub-only configuration options**. These settings cannot be changed in a later request and must be set when creating the Hub group. ``` POST /api/v1/tiscale-hub-group/create/ ``` Creates a Hub group. ##### Request body ```json5 { "name": "string", // name of the Hub group "primary_router_id": "string", // ID for the primary Hub (arbitrary string) "primary_host": "string", // name, not URL "fallback_router_id": "string", // also arbitrary string "fallback_host": "string", // name, not URL "vrrp_enabled": false, // enable/disable sharing a virtual IP address "virtual_router_id": 1, // ID of the virtual Hub "auth_pass": "string", // VRRP authentication password "different_networks": true, // whether the Hubs are inside different networks "virtual_ipaddress": "string", // virtual Hub IP address "api_worker_token": "string", // file analysis token "srv_available_worker_token": "string", // service availability check token "srv_worker_token": "string", // operational stats token } ``` After creating a Hub group, the next step is to save and apply its configuration. The steps are the same as for regular configuration groups, and the only difference is the endpoint. **Hub-only parameters** (i.e. those relating exclusively to Hub groups, and not regular configuration groups) **must be set upon Hub group creation**. ``` PUT /api/v1/tiscale-hub-group/save/{group_name}/ ``` Save Hub group configuration. This endpoint will overwrite the existing Hub Group configuration with the values sent in the `content` request parameter, so any existing Hub Group settings will be discarded before the request values are written to the group. Path parameters: - `group_name`: name of the hub group Request body: ```json5 { "content": { "snmp__enable": true, "snmp__community": "^$", "snmp__trap_sink_enable": true, "snmp__trap_community": "^$" // ... } } ``` ``` PATCH /api/v1/tiscale-hub-group/save/{group_name}/ ``` This endpoint will override the existing Hub Group configuration with the values sent in the `content` request parameter, so any existing Hub Group settings not listed will be preserved, while those listed will be overwritten. Path parameters: - `group_name`: name of the hub group Request body: ```json5 { "content": { "snmp__enable": true, "snmp__community": "^$", "snmp__trap_sink_enable": true, "snmp__trap_community": "^$" // ... ``` ------------------------------------------------ ``` PUT /api/v1/tiscale-hub-group/update/{group_name}/ ``` Applies the configuration set in the previous step and resolves any settings mismatch by resetting any settings not configured in the Hub Group. This endpoint will not make any changes to the Hub Group appliance configuration stored on the Manager. This endpoint will apply Hub Group configuration from the Manager to the group appliances listed in the `applyconfiguration` request parameter, and resolve any configuration mismatch on those appliances. Path parameters: - `group_name`: name of the hub group Request body: ```json5 { "applyconfiguration": [ "string" // list of which group appliances will receive hub group configuration ], "primary_router_id": "string", // ID for the primary Hub (arbitrary string) "primary_host": "string" // hostname or IP address, not URL } ``` ------------------------------------------------ ``` PATCH /api/v1/tiscale-hub-group/update/{group_name}/ ``` Applies the configuration set in the previous step and resolves any settings mismatch by resetting any settings not configured in the Hub Group. This endpoint will not make any changes to the Hub Group appliance configuration stored on the Manager. This endpoint will apply Hub Group configuration from the Manager to the group appliances listed in the `applyconfiguration` request parameter, and resolve any configuration mismatch on those appliances. Path parameters: - `group_name` - name of the hub group Request body: ```json5 { "applyconfiguration": [ "string" // list of appliance names indicating which appliances in the group will receive hub group configuration ], "primary_router_id": "string", // ID for the primary Hub (arbitrary string) "primary_host": "string" // hostname or IP address, not URL } ``` The rest of the configuration steps (verifying, checking the timestamp etc.) are described in the chapter on [Standard configuration groups](#standard-configuration-groups). ### Individual configuration: Spectra Analyze It is also possible to configure only individual Spectra Analyze appliances (instead of through groups). ``` PUT /api/v1/appliances/content/{appliance_id}/save/ ``` Saves the entire configuration (but doesn't yet apply it). Path parameters: - `appliance_id`: ID of the appliance There are many configurable fields that you can send (the full list is in the reference on the Manager). While the request above uses PUT, it is also possible to use PATCH. All the parameters are the same, and the only difference is that PATCH allows partial saving (and PUT always replaces the entire configuration). After saving the configuration, you once again must apply it: ``` POST /api/v1/appliances/content/{appliance_id}/apply/ ``` Applies the uploaded configuration to Spectra Analyze. ### Set a timezone This section covers listing and setting the timezone of the Spectra Analyze, Hub, and Worker. List available timezones for the Spectra Analyze, Hub, and Worker. The response contains a list of timezones that can be set. ``` GET /api/v1/system/config/list-available-timezones/ ``` Response: ```json5 { "timezones": [ "string" ] } ``` Set the timezone of the Spectra Analyze, Hub, or Worker. ``` POST /api/v1/appliances/{appliance_id}/system/configure-timezone/ ``` Path parameters: - `appliance_id`: ID of the appliance ## Configure connectors ### Enable connector To enable a connector for a specific appliance, use the following endpoint. This action is supported only for appliances of type "Hub". ``` POST /api/v1/appliances/{id}/connectors/{connector_name}/ ``` Path parameters: - `id`: The appliance ID, which must be a string of digits (e.g., `123`). - `connector_name`: The name of the connector to enable. Must be one of the following: - `abusebox` - `azure-data-lake` - `fileshare` - `graph-api` - `s3` - `sftp` - `share-file` - `smtp` - `icap` ### Get connector information To retrieve information about a specific connector of an appliance, use the following endpoint. This action is supported only for appliances of type "Hub". ``` GET /api/v1/appliances/{id}/connectors/ ``` Path parameters: - `id`: The appliance ID, which must be a string of digits (e.g., `123`). For a complete list of available information options, response fields, and examples for each connector service, refer to the API Reference Guide on the Manager. ### Get connector configuration To retrieve the configuration for a specific connector of an appliance, use the following endpoint. This action is supported only for appliances of type "Hub". ``` GET /api/v1/appliances/{id}/connectors/{connector_name}/v1/config/ ``` Path parameters: - `id`: The appliance ID, which must be a string of digits (e.g., `123`). - `connector_name`: The name of the connector whose configuration should be returned. Must be one of the following: - `abusebox` - `azure-data-lake` - `fileshare` - `graph-api` - `s3` - `sftp` - `share-file` - `smtp` - `icap` For a complete list of available configuration options, response fields, and examples for each connector service, refer to the API Reference Guide on the Manager. ### Configure connector To configure a specific connector for an appliance, use the following endpoint. This action is supported only for appliances of type "Hub". ``` POST /api/v1/appliances/{id}/connectors/{connector_name}/v1/config/ ``` Path parameters: - `id`: The appliance ID, which must be a string of digits (e.g., `123`). - `connector_name`: The name of the connector that should be configured. Must be one of the following: - `abusebox` - `azure-data-lake` - `fileshare` - `graph-api` - `s3` - `sftp` - `share-file` - `smtp` - `icap` #### Request body For a complete list of available configuration options, response fields, and examples for each connector service, refer to the API Reference Guide on the Manager. ### Test connector connection To test the connection to a connector, use the following endpoint. This action is supported only for appliances of type "Hub". ``` POST /api/v1/appliances/{id}/connectors/{connector_name}/test-connection/ ``` Path parameters: - `id`: The appliance ID, which must be a string of digits (e.g., `123`). - `connector_name`: The name of the connector to test. Must be one of the following: - `abusebox` - `azure-data-lake` - `fileshare` - `graph-api` - `s3` - `sftp` - `share-file` - `smtp` - `icap` #### Request body For a complete list of available configuration options, response fields, and examples for each connector service, refer to the API Reference Guide on the Manager. ### Disable connector To disable a connector for a specific appliance, use the following endpoint. This action is supported only for appliances of type "Hub". ``` DELETE /api/v1/appliances/{id}/connectors/{connector_name}/ ``` Path parameters: - `id`: The appliance ID, which must be a string of digits (e.g., `123`). - `connector_name`: The name of the connector to disable. Must be one of the following: - `abusebox` - `azure-data-lake` - `fileshare` - `graph-api` - `s3` - `sftp` - `share-file` - `smtp` - `icap` ## Configure Central Logging Central Logging provides API endpoints for retrieving and managing data generated during file analysis. It supports access to antivirus scan results, classification outputs, detections overview, and processing statistics. The service enables exporting of classification and error data for analysis and reporting. It also provides operations for rescanning samples and retrieving file type group counts. In addition, Central Logging manages Deep Cloud Analysis hash allowlists and blocklists, allowing listing, adding, and removing SHA1 hashes that control which files are excluded from or forced into scanning. ### Deep Cloud Analysis Hash Allowlist The Deep Cloud Analysis Hash Allowlist is a list of SHA1 hashes that should be treated as safe and excluded from AV scanning. It provides endpoints to retrieve, add, and remove allowlisted hashes, ensuring known safe files are skipped during analysis. #### Retrieve SHA1 hashes on allowlist Retrieves SHA1 hashes currently on the Deep Cloud Analysis allowlist, which are considered safe and are excluded from scanning. ``` GET /api/central-logging/v1/multiscan/hashes/ ``` #### Add SHA1 hashes to allowlist Add one or more SHA1 hashes to the Deep Cloud Analysis hash allowlist. Files with these hashes are treated as safe and skipped from further scanning. Validates input and reports invalid hashes. ``` PATCH /api/central-logging/v1/multiscan/hashes/ ``` #### Remove SHA1 hashes from allowlist Remove one or more SHA1 hashes from the Deep Cloud Analysis hash blocklist. Hashes should be sent in the request body as an array of strings. The response reports which hashes were successfully deleted, not found, or invalid. ``` DELETE /api/central-logging/v1/multiscan/hashes/ ``` ### Deep Cloud Analysis Hash Blocklist The Deep Cloud Analysis Hash Blocklist is a list of SHA1 hashes that should be treated as unsafe and included in AV scanning. It provides endpoints to retrieve, add, and remove blocked hashes, ensuring known unsafe files are scanned during analysis. #### Retrieve SHA1 hashes on blocklist Retrieves SHA1 hashes currently on the Deep Cloud Analysis hash blocklist, which are excluded from further analysis. ``` GET /api/central-logging/v1/multiscan/hashes/ ``` #### Add SHA1 hashes to blocklist Add one or more SHA1 hashes to the Deep Cloud Analysis hash blocklist. Validates input and reports invalid hashes. ``` PATCH /api/central-logging/v1/multiscan/hashes/ ``` --- ## Spectra Detect Prometheus Metrics API — GET /metrics and honor_labels Setup The Metrics API provides an interface for collecting Prometheus-compatible metrics from Spectra Detect appliances. Users can scrape this endpoint to monitor the health and performance of the appliance. ## Prometheus `honor_labels` Configuration When configuring your Prometheus server to scrape metrics from Spectra Detect appliances, we recommend enabling the `honor_labels` setting in your Prometheus scrape configuration. By default, Prometheus renames conflicting labels scraped from targets to avoid collisions with its own internally generated labels (like `instance` and `job`). For example, a target's `job` label might be renamed to `exported_job`. However, Spectra Detect provides certain labels (e.g., `job`) within its exposed metrics that are essential for proper identification and data filtering. Setting `honor_labels: true` in your Prometheus scrape configuration for Spectra Detect targets will instruct Prometheus to preserve the original label names exposed by the appliance. Here's an example of a Prometheus scrape configuration snippet with `honor_labels` enabled: ```yaml scrape_configs: - job_name: 'spectra-detect' honor_labels: true static_configs: - targets: [':'] ``` ## Metrics API response status codes - 200 - The request has succeeded. - 404 - The server has not found anything matching the Request-URI. - 500 - The server encountered an unexpected condition which prevented it from fulfilling the request. - 503 - The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. ## Get Prometheus metrics ``` GET /metrics ``` ### Description Returns the current system and application metrics in a Prometheus-compatible format. This endpoint can be scraped by a Prometheus server to collect data about the appliance. ### Request format **Request examples** ```bash curl -X GET https:///metrics ``` ```python url = "https:///metrics" response = requests.request("GET", url) print(response.text) ``` ### Response format A successful request returns the `200 OK` status code with the metrics in the Prometheus exposition format in the response body. Each appliance returns metrics specific to the services it hosts. All appliances expose `node_*` metrics. Specifically, Hub appliances only provide `node_*` metrics. Manager appliances expose `node_*` and `rabbitmq_*` metrics. Worker appliances provide `node_*`, `rabbitmq_*`, and `rl_*` metrics. ### Aggregating metrics per appliance While the metrics endpoint exposes raw metrics for each internal service (e.g., `job="postprocessor"`, `job="receiver"`), individual service metrics may show a value of 0. To view meaningful metrics for a particular appliance, aggregate the metrics across all services on that appliance using the `sum()` function in your Prometheus queries. For example, to get the total processed bytes for a Worker appliance: ```promql sum(rl_processed_bytes_total{instance="10.200.185.215"}) ``` Replace `10.200.185.215` with the IP address of the target appliance. **Response examples** ``` node_memory_Mapped_bytes{job="node"} 4.29580288e+08 node_network_receive_compressed_total{device="ens192",job="node"} 0 node_network_receive_compressed_total{device="lo",job="node"} 0 node_disk_writes_merged_total{device="dm-0",job="node"} 0 node_disk_writes_merged_total{device="dm-1",job="node"} 0 node_disk_writes_merged_total{device="sda",job="node"} 87857 node_disk_writes_merged_total{device="sr0",job="node"} 0 node_disk_writes_merged_total{device="sr1",job="node"} 0 rabbitmq_detailed_queue_messages_ready{vhost="rl-detect",queue="tiscale.hagent_retry",job="rabbitmq"} 0 rabbitmq_detailed_queue_messages_ready{vhost="rl-detect",queue="tiscale.hagent_error",job="rabbitmq"} 0 rabbitmq_detailed_queue_messages_ready{vhost="rl-detect",queue="tiscale.hagent_input",job="rabbitmq"} 0 rabbitmq_detailed_queue_messages_ready{vhost="rl-detect",queue="tiscale.hagent_url",job="rabbitmq"} 0 rabbitmq_detailed_queue_messages_ready{vhost="rl-detect",queue="tiscale.hagent_result",job="rabbitmq"} 0 rl_processed_bytes_total{job="postprocessor"} 0 rl_processed_files_total{job="postprocessor"} 0 rl_submitted_bytes_total{job="postprocessor"} 0 rl_submitted_files_total{job="postprocessor"} 0 rl_unpacked_bytes_total{job="postprocessor"} 0 rl_unpacked_files_total{job="postprocessor"} 0 rl_processed_bytes_total{job="receiver"} 0 rl_processed_files_total{job="receiver"} 0 rl_submitted_bytes_total{job="receiver"} 0 rl_submitted_files_total{job="receiver"} 0 rl_unpacked_bytes_total{job="receiver"} 0 rl_unpacked_files_total{job="receiver"} 0 ``` --- ## Spectra Detect Worker Service API — System Status and Processing Queue The Service API provides an interface for controlling processing-related functionalities on the Worker. Users can check the status and availability of the Worker system, and retrieve file processing statistics to observe changes in system performance. ## Service API Authorization The Worker can be configured to require an authorization token to be present in every API request. The authorization token is a string of alphanumeric characters, between 30 and 60 characters in length. The authorization token should be provided in the HTTP `Authorization` header. The value of this field must be in this format: ``` Token exampletoken ``` where `exampletoken` should be replaced with the user's own authorization token string. If the Worker is configured not to require an authorization token, the `Authorization` header field will be ignored. ```bash curl https://tiscale-worker01/srv/tiscale/v1/sysinfo -H 'Authorization: Token exampletoken' ``` ## Service API Request Format The base URL for the service API is in the following format: ``` https://{server}/srv/tiscale/v1 ``` ## Service API response status codes - 200 - The request has succeeded. - 400 - The request could not be understood by the server due to malformed syntax or missing parameters. - 401 - The request requires authorization from the user. - 403 - Authorization has been provided in the request, but it is not valid. - 404 - The server has not found anything matching the Request-URI. - 500 - The server encountered an unexpected condition which prevented it from fulfilling the request. - 503 - The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. - In case the Spectra Intelligence upload limit quota was exceeded, the response will contain the "Paused - TiCloud quota exceeded" error message. To continue analyzing files, users can temporarily disable Spectra Intelligence, wait until the daily quota renews, or contact ReversingLabs support to increase their daily upload limit. When using the Quotas API endpoints on a Worker instance without a configured Spectra Intelligence account, the response code will be 503, and the error message "TiCloud is not configured." ## Get the current system information ``` GET /srv/tiscale/v1/sysinfo ``` ### Description Returns current system information, indicating the status of processing-related components on the Worker instance. ### Request format **Request examples** ```bash curl -X GET https://tiscale-worker01/srv/tiscale/v1/sysinfo -H 'Authorization: Token exampletoken' ``` ```python url = "https://tiscale-worker01/srv/tiscale/v1/sysinfo" headers = {'Authorization': 'Token exampletoken'} response = requests.request("GET", url, headers=headers) print(response.text) ``` ### Response format A successful request returns the `200 OK` status code with the `sysinfo` structure in the response body. **Response example** ```json { "load_average": [ 0.47, 0.27, 0.28 ], "queue_size": 0, "max_queue_name": "tiscale_hagent_input", "max_queue_size": 0, "hagent_running": true, "scratch_space": 3, "root_space": 3 } ``` **Response fields** | Field Name | Description | Data Type | | ---------------- | ------------------------------------------------------------ | --------- | | `load_average` | Average system load for the past 1, 5, and 15 minutes. | float | | `queue_size` | Amount of files in the input queue. | integer | | `max_queue_name` | Name of queue which currently holds the most files. | string | | `max_queue_size` | Number of files in the currently largest queue. | integer | | `hagent_running` | This field is no longer supported. For the time being, it always returns the `true` value. | boolean | | `scratch_space` | Indicates the percentage of used disk space on the partition holding files to be processed and reports for processed files. | float | | `root_space` | Indicates the percentage of used disk space on the root partition. | float | ## Control the Worker operating mode ### Check current operating mode status ``` GET /srv/tiscale/v1/mode ``` Returns whether the Worker is currently paused or not. Setting the `paused` flag to `true` will cause the Worker to decline all new file submissions until the flag is unset. The flag can be used to put a Worker into maintenance mode or to remove it from a pool of Workers. #### Request format **Request example** ```bash curl -X GET https://tiscale-worker01/srv/tiscale/v1/mode -H 'Authorization: Token exampletoken' ``` #### Response format The response indicates whether the Worker instance is paused (true) or not (false). If the response equals `"paused": true` the Worker is considered unavailable, and is not accepting any new file submissions. **Response example** ```json { "paused": false } ``` ### Change operating mode status ``` PUT /srv/tiscale/v1/mode ``` The operating mode of the Worker instance can be changed by sending a PUT request to the same Service API endpoint. The value of the `Content-Type` HTTP header field must be set to `application/json`. The data supplied in the request must be a JSON object with one or more fields. #### Request format **Request example** Set Worker operating mode to paused ```bash curl -X PUT https://tiscale-worker01/srv/tiscale/v1/mode -H 'Authorization: Token exampletoken' -H 'Content-Type: application/json' -d '{"paused": true}' ``` #### Response format A successful request returns the `200 OK` status code with an empty response body. If the `Content-Type` HTTP header in the request is not set to `application/json`, the response contains an error message "Invalid POST data: expected JSON object". Users experiencing this error message should submit a new request with the incorrect HTTP header properly configured. ## Check Worker liveness ``` GET /api/status/v1/liveness ``` ### Description Checks whether the Worker service is running and responsive. This endpoint is designed for external monitoring systems and uptime checks. Unlike `/srv/tiscale/v1/available`, this endpoint reports actual service availability and does not return error status codes when cloud quota is exhausted or the license has expired. ### Request format **Request example** ```bash curl -X GET https://tiscale-worker01/api/status/v1/liveness ``` ### Response format **Response codes:** - `200 OK`: The Worker service is up and running. - `429 Too Many Requests`: The Worker is overloaded (queue is over the configured threshold). The service is alive but temporarily unable to accept new work. - `503 Service Unavailable`: The Worker service is down or in an unhealthy state. **Response example (200 OK):** ```json { "message": "", "status": "OK" } ``` This endpoint is recommended for external monitoring systems to track service uptime. ## Check Worker readiness ``` GET /api/status/v1/readiness ``` ### Description Checks whether the Worker is ready to process files. This endpoint performs comprehensive health checks including license status, disk and queue thresholds, cloud quota, database connectivity, and service status. ### Request format **Request example** ```bash curl -X GET https://tiscale-worker01/api/status/v1/readiness ``` ### Response format **Response codes:** - `200 OK`: The Worker is ready to process files. - `503 Service Unavailable`: The Worker is not ready due to one or more issues (unlicensed, paused, quota exceeded, unhealthy state, or connectivity problems). **Response example (200 OK):** ```json { "message": "", "status": "OK" } ``` ## Check file processing availability ``` GET /srv/tiscale/v1/available ``` ### Description Checks whether the Worker instance is available for processing new files. ### Request format **Request example** ```bash curl -X GET https://tiscale-worker01/srv/tiscale/v1/available -H 'Authorization: Token exampletoken' ``` ### Response format If the Worker is available, it returns the status code `200 OK` with no message in the response body. If the instance is not available (`503 Service Unavailable`), it's either in an error state, or paused. There are two types of *paused* states: 1. Operating mode set to `paused`. In this case, the response body contains the following message: `paused`. See [control the Worker operating mode](#control-the-worker-operating-mode). 2. Paused due to an exceeded Spectra Intelligence quota. In this case, the response body contains the following message: `paused - TiCloud quota exceeded`. Depending on how your quota is configured (number of uploads vs. total byte size uploaded; daily vs. monthly), either wait for the quota to renew automatically or contact support to modify your quota. Alternatively, remove the Spectra Intelligence account altogether (this will disable AV scans in the cloud - only static analysis will be available). If the Worker is in an error state, for example due to a problem with system resources, the response body contains a `not healthy` message. ## Get file processing statistics ``` GET /srv/tiscale/v1/stats ``` ### Description Returns the file processing statistics for the Worker instance. ### Request format **Request example** ```bash curl https://tiscale-worker01/srv/tiscale/v1/stats -H 'Authorization: Token exampletoken' ``` ### Response format The format of the data returned is described by the `stats` structure. ``` stats := { "timeframes" : ["1m", "10m", "1h", "8h", "1d"], "files_submitted" : [, , , , ], "bytes_submitted" : [, , , , ], "files_processed" : [, , , , ], "bytes_processed" : [, , , , ], "files_unpacked" : [, , , , ], "bytes_unpacked" : [, , , , ], "mean_queue_size" : [, , , , ], } ``` Statistics information is accrued and displayed for multiple time windows. The `timeframes` field lists shorthand names for those time windows (1m - one minute, 1h - one hour, etc.). - `ųpload_success` shows the number of successful file uploads to the Worker. - `upload_failure` shows the number of failed file uploads to the Worker. - `s3_unpacked_success` shows how many children samples successfully unpacked to the S3 bucket. - `s3_report_upload` shows how many reports were successfully uploaded to the S3 bucket. - `s3_upload` shows the number of files that were successfully uploaded to the S3 bucket. - `s3_unpacked_failure` shows how many children samples failed to unpack to the S3 bucket. - `sns_failure` shows the number of Amazon SMS messages that failed to deliver. - `max_mean_queue_name` shows the names of queues with the largest mean amount of tasks in that timeframe. - `general_service_failure` shows the amount of service errors (5xx server errors). - `callback_failure` shows how many reports failed to upload to the Callback server. - `files_submitted` shows the number of files that were submitted for processing. - `mean_queue_size` shows the mean value of the input queue size for each of the time windows. - `general_user_failure` shows the number of errors caused by the user, for example by malformed API requests (4xx client errors). - `files_processed` and `bytes_processed` fields show the number of files/bytes that were successfully processed. - `splunk_failure` shows how many reports failed to upload to Splunk. - `s3_upload_failure` shows the number of unsuccessful uploads to the S3 bucket. - `bytes_submitted` shows the total number of bytes of all files that were submitted. - `splunk_success` shows how many reports were successfully uploaded to Splunk. - `callback_success` shows how many reports were successfully uploaded to the Callback server. - `not_available` is triggered when the Worker is in an unhealthy state (for example, disk usage is above the set threshold). - `sns_success` shows the number of successfully delivered Amazon SMS messages. - `max_mean_queue_size` shows the mean amount of tasks for queues listed in `max_mean_queue_name`. - `s3_report_upload_failure` shows how many reports failed to upload to the S3 bucket. - `files_unpacked` and `bytes_unpacked` fields show the number of files/bytes that were extracted (unpacked) from samples that were uploaded for processing. - `file_processing_failure` shows the number of files that the service failed to analyze. **Response example** A successful request returns the `200 OK` status code with the list of processing statistics fields and their values in the response. ```json { "upload_success": [ 7, 15, 15, 15, 15 ], "timeframes": [ "1m", "10m", "1h", "8h", "1d" ], "upload_failure": [ 0, 0, 0, 0, 0 ], "s3_unpacked_success": [ 0, 0, 0, 0, 0 ], "s3_report_upload": [ 0, 0, 0, 0, 0 ], "s3_upload": [ 0, 0, 0, 0, 0 ], "s3_unpacked_failure": [ 0, 0, 0, 0, 0 ], "sns_failure": [ 0, 0, 0, 0, 0 ], "max_mean_queue_name": [ "tiscale.hagent_input", "tiscale.hagent_input", "tiscale.hagent_input", "tiscale.hagent_input", "tiscale.hagent_input" ], "general_service_failure": [ 0, 0, 0, 0, 0 ], "callback_failure": [ 0, 0, 0, 0, 0 ], "bytes_processed": [ 142273712, 304872240, 304872240, 304872240, 304872240 ], "files_submitted": [ 7, 15, 15, 15, 15 ], "mean_queue_size": [ 0.625, 0.08908045977011493, 0.014432029795158285, 0.001795228167709057, 0.0008703953279424976 ], "general_user_failure": [ 0, 0, 0, 0, 0 ], "files_processed": [ 7, 15, 15, 15, 15 ], "splunk_failure": [ 0, 0, 0, 0, 0 ], "bytes_unpacked": [ 303859038, 651126510, 651126510, 651126510, 651126510 ], "s3_upload_failure": [ 0, 0, 0, 0, 0 ], "bytes_submitted": [ 142273712, 304872240, 304872240, 304872240, 304872240 ], "splunk_success": [ 0, 0, 0, 0, 0 ], "callback_success": [ 0, 0, 0, 0, 0 ], "not_available": [ 0, 0, 0, 0, 0 ], "sns_success": [ 0, 0, 0, 0, 0 ], "max_mean_queue_size": [ 0.625, 0.08908045977011493, 0.014432029795158285, 0.001795228167709057, 0.0008703953279424976 ], "s3_report_upload_failure": [ 0, 0, 0, 0, 0 ], "files_unpacked": [ 56, 120, 120, 120, 120 ], "file_processing_failure": [ 0, 0, 0, 0, 0 ] } ``` ## Download support logs ``` GET /api/tiscale/v1/support --output hub_support_logs.zip ``` ### Description Download the support logs from Hub or Worker appliances. The contents of the archive are identical to the contents of the archive generated by clicking the *Download Logs* button on the appliance status page. **Request example** ```bash curl https://detect-hub01/api/tiscale/v1/support --output hub_support_logs.zip ``` --- ## Spectra Detect Usage API — File Submission, Task Status, and Report Retrieval The usage API is the main interface for interacting with the file analysis engine. With this API, users can submit files for analysis, check the status of file processing tasks, retrieve analysis reports when file processing is complete, and delete file processing tasks. ## Worker API Authorization The Worker can be configured to require an authorization token to be present in every API request. The authorization token is a string of alphanumeric characters, between 30 and 60 characters in length. There are several different tokens, depending on the accessed endpoint. The authorization token should be provided in the HTTP `Authorization` header. The value of this field must be in this format: ``` Token exampletoken ``` where `exampletoken` should be replaced with the user's own authorization token string. If the Worker is configured not to require an authorization token, the `Authorization` header field will be ignored. ```bash curl https://tiscale-worker01/api/tiscale/v1/task/42 -H 'Authorization: Token exampletoken' ``` ## API Request Format The base URL for the Spectra Detect API is in the following format: ``` https://{server}/api/tiscale/v1 ``` ## API Response Status Codes | Code | Name | Description | | ---- | --------------------- | ------------------------------------------------------------ | | 200 | OK | The request has succeeded. | | 400 | Bad Request | The request could not be understood by the server due to malformed syntax or missing parameters. | | 401 | Unauthorized | The request requires authorization from the user. | | 403 | Forbidden | Authorization has been provided in the request, but it is not valid. | | 404 | Not Found | The server has not found anything matching the Request-URI. | | 500 | Internal Server Error | The server encountered an unexpected condition which prevented it from fulfilling the request. | | 503 | Service Unavailable | The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. In case the Spectra Intelligence upload limit quota was exceeded, the response will contain the "Paused - TiCloud quota exceeded" error message. To continue analyzing files, users can temporarily disable Spectra Intelligence, wait until the daily quota renews, or contact ReversingLabs support to increase their daily upload limit. When using the [Quotas API](#quotas-api) endpoints on a Worker instance without a configured Spectra Intelligence account, the response code will be 503, and the error message "TiCloud is not configured." | ## License appliances ### Return License Information Returns the current status and detailed licensing information for the appliance. If there is no machine ID, **it will be generated**, and the response will contain information associated with this newly generated machine ID. #### Request ``` GET /api/license/v1/ ``` ```bash curl https://tiscale-worker-01/api/license/v1/ -H 'Authorization: Token exampletoken' ``` #### Response ```json5 { "product" : "string", // always "tiscale-worker" "machineId": "string", // ID of the appliance "version": 1, // version of the license file "serialNumber": "string", // serial number of the license file applied to the Worker "licenseType": "string", // can either be to WILDCARD or REGULAR "company": "string", // company name that the license file is provided for "expiryDate": "string", // date when the license will expire (YYYY-MM-DDTHH:mm:ssZ) "creationDate": "string", // starting date of the license "status": "string" // licensing status } ``` Description of fields: - **licenseType** can either be equal to `WILDCARD` or `REGULAR`: - `WILDCARD` licenses are independent of the machine ID and can last for a maximum of 45 days; trial licenses belong to this category - `REGULAR` licenses depend on the machine ID and have a custom duration - **company** is the company name that the license file is provided for; it will be equal to the Spectra Intelligence username in case the cloud is used for licensing - status can be one of the following: - `VALID` - `ONLINE` - `INVALID` - `EXPIRED` - `NO_LICENSE` - `NO_MACHINE_ID` ### Return License Status Returns the current licensing status of the appliance. #### Request ``` GET /api/license/v1/status/ ``` ``` curl https://tiscale-worker-01/api/license/v1/status/ -H 'Authorization: Token exampletoken' ``` #### Response ```json { "status": "string" } ``` Status can be one of the following: - `VALID` - `ONLINE` - `INVALID` - `EXPIRED` - `NO_LICENSE` - `NO_MACHINE_ID` ### Generate Machine ID Creates a new machine ID. #### Request ``` POST http(s)://{server}/api/license/v1/machine-id/generate ``` The request body is empty. ``` curl -X POST https://tiscale-worker-01/api/license/v1/machine-id/generate -H 'Authorization: Token exampletoken' ``` #### Response - 200 OK `{ "machineId": "string" }` - 500 Internal Server Error `{ "message": "string" }` ### Upload License File Uploads a license file to the appliance. Not required for licenses configured using Spectra Intelligence. #### Request ``` PUT /api/license/v1/upload ``` Content type: `multipart/form-data` Example: ```bash curl -X PUT https://tiscale-worker-01/api/license/v1/upload -H 'Authorization: Token exampletoken' -F 'file=@example_license_file' ``` #### Response ```json { "message": "string" } ``` The `message` will differ depending on the response code: - 201: "License is applied" - 200: "License is already applied" - 400: returns an error message ## Upload a file for processing **Direct upload** File uploads can be performed synchronously or asynchronously. ``` POST /api/tiscale/v1/upload ``` Uploads a file for processing and immediately returns a `task_url` that can be queried later to retrieve the full analysis report once processing is complete. --- ``` POST /api/tiscale/v1/upload/sync ``` Uploads a file for processing and waits until processing is complete, returning the full analysis report directly in the response. Accepts the same parameters as the [Get information about a processing task](#get-information-about-a-processing-task-taskinfo) endpoint. **Upload from URL** ``` POST /api/tiscale/v1/submit-url ``` **Upload a container image** ``` POST /api/tiscale/v1/submit-container-image ``` ### Description Submits a file, a URL-based file, or container image registry reference to the Worker or Hub for analysis; container images are pulled directly from the specified registry. If the request is sent to a Hub with one or more connected Workers, the file(s) will be forwarded to the connected Workers using the round-robin scheduling algorithm, and the API response will include a link to the file processing task on the Worker. Analysis reports cannot be obtained from a Hub. To get the analysis report for each submitted file, users should send a request directly to the Worker using the link to the processing task returned in the API response. See [Get information about a processing task](#get-information-about-a-processing-task-taskinfo) for more information. If you are uploading a file directly, use the `file` parameter and set the request payload type to `multipart/form-data`. Else, if you are uploading a file via a URL, use the `url` parameter and set the request payload type to `application/json`. If you are uploading a container image, send JSON with an `image` field (e.g. `{ "image": "alpine:3.21"}`). Optional fields `image_platform` and `credential_helper_authentication` let you choose a specific architecture or supply registry credentials. Workers can be configured to require an authorization token in every API request. In that case, the `Authorization` header with a token must be included in the request. Users can provide an optional `X-TiScale-Token` header with a string that can be used to filter file processing tasks. Users can also submit optional fields in the file upload request: - `user_data` *type: string | format: JSON* This parameter can be used to store metadata in the analysis reports sent to Splunk, or to enable additional report-related options. The report doesn't include the contents of `user_data`. - `custom_data` *type: string | format: JSON* This parameter can be used to append any additional information about a file to its analysis report. This allows users to filter, reference, or distinguish samples and their analysis reports more easily. The value provided in the parameter is saved and returned in all reports on the Worker appliance - in the Worker API response and the Callback response, in reports sent to Splunk, and in the reports saved to AWS S3 (if enabled). Note that it is not included in reports when the file has not been processed yet. If an `integrity_check_hash` field is included in the `custom_data` field, an integrity check of the uploaded file is triggered on the Worker. If the SHA1 hash of the uploaded sample doesn't match the `integrity_check_hash` field sent by the client, the sample will be discarded with the following error message: "ERROR: Upload: failed to accept upload integrity check failed, calculated hash XX differs from the one passed in custom data YY." This field is optional and is implemented in the S3 connector. If a container image is submitted, the Worker automatically appends an `image_metadata` object to the `custom_data` section of the analysis report. This metadata includes resolved values such as the image reference and platform. If `custom_data` already contains an `image_metadata` field, it will not be overwritten. If the structure of `custom_data` isn't valid JSON, it will not be included in the report. **File Source Tag** The `custom_data` parameter supports a `source_tag` field to track file origin and provide end-to-end traceability. This field is included in the analysis report and displayed as a filterable column in the Detections Overview table on the Spectra Detect Manager dashboard. The `source_tag` field accepts custom metadata to identify the line of business, department, or other organizational unit from which the file originated. Example: ```json { "source_tag": "ExampleTag" } ``` When submitting files via the API, include the `source_tag` field in the `custom_data` parameter: ```bash curl -X POST https://tiscale-worker-01/api/tiscale/v1/upload \ -H 'Authorization: Token exampletoken' \ -F file=@sample.pdf \ -F 'custom_data={"source_tag": "ExampleTag"}' ``` For ICAP submissions, use the `X-System-Tag` header to automatically populate the `source_tag` field. See the [ICAP Server configuration](../Config/AnalysisInput.md#icap-server) section for details. To get the analysis report for each submitted file, users should send a request to the [Get information about a processing task](#get-information-about-a-processing-task-taskinfo) endpoint. **Note: **Data retention**** All processed files are deleted immediately after processing. The reports are retained on the Workers for up to 90 minutes (default value), configurable by the "Task Age Limit" setting in the Hub Configuration Group under Worker Configuration - Cleanup. If a file is queued but not processed within the time configured by the "Unprocessed Task Limit", the processing task will be canceled, and the file will be deleted. However, the record of the unsuccessful task will still be present in the database for 1440 minutes (24 hours) before being deleted, as configured by the "File Age Limit" setting. ### Request format **Request parameters** | Parameter Name | Description | Required | |-------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------| | `file` | Path to the file that should be uploaded to the Worker for processing. Mutually exclusive with the `url` and `image` parameters. Used for the `upload` endpoint. | One of `file`, `url`, or `image` | | `url` | URL to the file that should be uploaded to the Worker for processing. Mutually exclusive with the `file` and `image` parameters. Used for the `submit-url` endpoint. | One of `file`, `url`, or `image` | | `image` | Container image reference (e.g. `busybox`, `alpine:3.21`). Mutually exclusive with `file` and `url`. Used for the `submit-container-image` endpoint. | One of `file`, `url`, or `image` | | `image_platform` | Target platform for the image (e.g. `linux/amd64`, `windows/amd64:10.0.26100.3476`). If not provided, the Worker uses its host platform. | Optional | | `credential_helper_authentication` | Registry credentials using the Docker credential helper format:`{ "Username": "your-user", "Secret": "your-token" }`. Only needed for private registries. | Optional | | `custom_data` | Accepts any user-defined data as a JSON-encoded payload. This data is included in all file analysis reports (Worker API, Callback, AWS S3 and Splunk, if enabled) in its original, unchanged form. The `custom_data` field will not be returned in the `taskinfo` endpoint response if the file has not been processed yet. | Optional | | `user_data` | Additional JSON-encoded payload can be supplied in a request in the `user_data` field. Portions of that data will be used in parts of the processing pipeline. The contents of `user_data` are not returned in the analysis report, except the `network` object in reports sent to Splunk (if configured). Supported objects are `network`, `callback`, `deep_cloud_analysis`, as well as the `password_list` array. | Optional | | `hash` | Optional parameter used for integrity verification of the downloaded file. The supported hash values are: `MD5`, `SHA1`, `SHA256`, or `SHA512`. The proper algorithm is determined automatically based on the hash length: `MD5` (16 bytes), `SHA1` (20 bytes), `SHA256` (32 bytes), `SHA512` (64 bytes). | Optional | | `custom_authorization_header` | Specifies a custom authorization header to be included in the HTTP request. Provide the header name and its corresponding value in the `custom_authorization_header` field. For example, to include an authorization token in the `Custom-Token` header, use `"custom_authorization_header": {"Custom-Token": "token_value"}`. If not provided, no custom header is added to the HTTP request. | Optional | `user_data.password_list` If the submission is a password-protected archive, provide a password list to be used to decrypt the archive. ```json { "password_list": ["pass", "pass2", "pass3"] } ``` `user_data.network` Optional network-related metadata can be supplied in the file upload request. If the Worker is configured to integrate with Splunk, this network metadata will also be sent to Splunk. ```json { "network": { "src_ip": "10.1.10.42", "src_port": 10621, "dst_ip": "10.1.10.212", "dst_port": 8080 } } ``` `user_data.callback` If the Callback feature is enabled and configured on the Worker instance, file analysis reports will be automatically sent to the configured Callback server. By default, only the medium (summary) report is sent to the Callback server. However, users can include an optional `callback` JSON object in the `user_data` field of the file upload request to have an extended analysis report delivered to the Callback server. The prerequisite is that the Callback feature is enabled on the Worker instance. ```json { "callback": { "report_type": "large", "view": "flat", "url": "http://example.com:8080" } } ``` By default, the report returned by the Worker is a **medium (summary) report**. Setting it to `large` will return the full report, whereas `small` and `medium` are default options that filter out certain fields. The `view` parameter can be used to apply a custom view to the report. For example, to enable sending an extended analysis report for the submitted file to the Callback server, provide the `view` parameter with the `extended` value. The `url` parameter specifies to which URL the report will be sent. If this is not specified, the report is sent to the default URL configured. If there is no `callback` object in the file upload request, the Worker uses preconfigured values. `user_data.deep_cloud_analysis` If the Deep Cloud Analysis is enabled in the **Administration > Spectra Detect Manager > Dashboard Configuration** section, the `deep_cloud_analysis` object can be included in the `user_data` field of the file upload request. The `deep_cloud_analysis` object contains optional parameters that can be used for Deep Cloud Analysis. ```json { "deep_cloud_analysis": { "full_scan": true, "skip_av_scan": true, "skip_wait_for_av_results": true, "wait_timeout_in_minutes": 30, "skip_unpacked_files": true, "rescan_enabled": true, "rescan_threshold_in_days": 0, "hash_block_list": [ "SHA1 hash", "SHA1 hash", "SHA1 hash" ] } } ``` - `full_scan`: Perform AV (re)scan of the container and unpacked samples ignoring the XREF record scan date. If set to `true`, it overrides any other configured setting except for the `hash_block_list`. The file size limit is 2.2 GB. - `skip_av_scan`: Skip the AV scan and process the sample. Sample upload needs to be enabled. - `skip_wait_for_av_results`: Skip waiting for the AV results. Has to be globally enabled in the **Central Configuration > Spectra Intelligence > Deep Cloud Analysis** section. - `wait_timeout_in_minutes`: Timeout in minutes for waiting for the AV results. - `skip_unpacked_files`: Skip AV analysis of unpacked files. Has to be globally enabled in the **Central Configuration > Spectra Intelligence > Deep Cloud Analysis** section. - `rescan_enabled`: Modify global rescan for current analysis. - `rescan_threshold_in_days`: Modify rescan threshold in days for current analysis. `0` indicates to always re-scan. - `hash_block_list`: An extension of the global hash block list defined via the Spectra Detect APIs. For more information on the Deep Cloud Analysis pipeline, refer to the [Deep Cloud Analysis Pipeline](../Usage/Analysis.md#deep-cloud-analysis-pipeline-diagram) diagram. **Request examples** ```bash curl -X POST https://tiscale-worker-01/api/tiscale/v1/upload -H 'Authorization: Token exampletoken' -F file=@Classification.zip -F 'user_data={"callback": {"view": "flat", "report_type": "large", "url": "https://example.com:8080"}, "password_list": ["pass","pass2","pass3"]}' curl -X POST https://tiscale-worker-01/api/tiscale/v1/upload -H 'Authorization: Token exampletoken' -F file=@Classification.pdf -F 'custom_data={ "file_source" : { "uploader" : "malware_analyst", "origin" : "example" }}' curl -X POST https://tiscale-worker-01/api/tiscale/v1/upload -H 'Authorization: Token exampletoken' -H 'X-TiScale-Token: Token custom' -F file=@Classification.pdf curl -X POST https://tiscale-worker-01/api/tiscale/v1/submit-url -H 'Authorization: Token exampletoken' -H 'Content-Type: application/json' -d '{"url": "https://example.com/file"}' curl -X POST https://tiscale-worker-01/api/tiscale/v1/submit-url -H "Content-Type: application/json" -d '{"url": "https://example.com/file", "custom_authorization_header": {"Custom-Token": "token_value"}}' # Upload a container image – Docker Hub, default tag (latest) curl -X POST https://tiscale-worker-01/api/tiscale/v1/submit-container-image \ -H 'Authorization: Token exampletoken' \ --json '{ "image": "busybox" }' # Upload with platform and tag curl --json '{ "image": "alpine:3.21.3", "image_platform": "linux/arm64/v8" }' \ https://tiscale-worker-01/api/tiscale/v1/submit-container-image # Upload from private registry with credentials curl --json '{ "credential_helper_authentication": { "Username": "", "Secret": "" }, "image": "private.registry.com/image:1.0.0", "image_platform": "linux/amd64" }' https://tiscale-worker-01/api/tiscale/v1/submit-container-image ``` For additional workflows, such as scanning entire directories, see our [SDK documentation and its cookbook](https://github.com/reversinglabs/reversinglabs-sdk-cookbook). ### Response format The response is a JSON-encoded object with a `task_url` field which contains the URL of the processing task started by the file or URL submission. If the task URL in the response is missing the `server` component, the same server value used for the file submission request should be used to access the task URL. The response will also contain a link to an S3 report if an output report bucket is configured. **Response example** ```json { "task_url": "https://tiscale-worker01:8080/api/tiscale/v1/task/1" } ``` ## List processing tasks ``` GET /api/tiscale/v1/task?age=700&token=token ``` ### Description Lists processing tasks generated by file submission requests. When a file is submitted for analysis to a Worker, a file processing task is created and queued on the Worker server. **Note: **Data retention**** All processed files are deleted immediately after processing. The reports are retained on the Workers for up to 90 minutes (default value), configurable by the "Task Age Limit" setting in the Hub Configuration Group under Worker Configuration - Cleanup. If a file is queued but not processed within the time configured by the "Unprocessed Task Limit", the processing task will be canceled, and the file will be deleted. However, the record of the unsuccessful task will still be present in the database for 1440 minutes (24 hours) before being deleted, as configured by the "File Age Limit" setting. This endpoint returns a list of processing tasks that have not yet expired; that is, tasks for which it is still possible to retrieve the analysis report. Every processing task is assigned a unique ID, returned in the `task_id` response field. This ID can be used in requests to other Worker API endpoints. If the optional `age` parameter is supplied in the request, only tasks older than the specified number of seconds are returned in the response. Task age is calculated as being the difference between the current system timestamp and the timestamp of the task submission. Timestamps are specified in the Unix time format, as the number of seconds since 1970-01-01 00:00:00. When submitting files for analysis, users can add an optional `X-TiScale-Token` header with a custom string to the file upload request. That custom string can be provided as the value for the optional `token` parameter in the request to this endpoint. The response will list only the tasks for files submitted with the specified custom string. **These parameters should not be confused with the mandatory authorization token that must be provided in the header of every request.** ### Request format **Request parameters** | Parameter Name | Description | Required | | -------------- | ------------------------------------------------------------ | -------- | | `age` | Specify the number of seconds to filter processing tasks by age. When provided in the request, the API returns only those processing tasks that are older than the specified number of seconds. Task age is the difference between the current system time and the time when the file processing task was created. | Optional | | `token` | Specify a custom string to filter processing tasks, if there are any files that were uploaded with that custom string in the `X-TiScale-Token` header. | Optional | **Request examples** Listing all available tasks for files uploaded with a custom X-TiScale-Token string: ```bash curl -X GET 'https://tiscale-worker-01/api/tiscale/v1/task?token=custom' -H 'Authorization: Token exampletoken' ``` Listing all available tasks older than 500 seconds: ```bash curl -X GET 'https://tiscale-worker-01/api/tiscale/v1/task?age=500' -H 'Authorization: Token exampletoken' ``` ```python url = "https://tiscale-worker-01/api/tiscale/v1/task" querystring = {"age":"500"} headers = {'Authorization': 'Token exampletoken'} response = requests.request("GET", url, headers=headers, params=querystring) print(response.text) ``` ### Response format If the request was successfully sent, the response includes a `200 OK` status code and the list of processing tasks for which it is still possible to retrieve the analysis report. **Response example** ```json [ { "submitted": 1481137280, "task_id": 27637, "processed": 1481137281 }, { "submitted": 1481137302, "task_id": 27638, "processed": 1481137303 } ] ``` **Response fields** | Field Name | Description | | ----------- | ------------------------------------------------------------ | | `submitted` | Timestamp indicating the time when the file was submitted for analysis, formatted as the number of seconds since 1970-01-01 00:00:00. | | `processed` | Timestamp indicating the time when the file was processed by the Worker, formatted as the number of seconds since 1970-01-01 00:00:00. | | `task_id` | Unique identification number assigned to each file processing task on the Worker instance. This task ID can be used in requests to other Worker API endpoints; for example, to retrieve the analysis report for a task, or to delete a task from Worker. To ensure backward compatibility with previous Worker versions, the task ID values are currently integers instead of strings. | ## Get information about a processing task {#get-information-about-a-processing-task-taskinfo} ``` GET /api/tiscale/v1/task/{task_id}?report_type={large}&view={flat} ``` ### Description Retrieves information about a completed file processing task. The `task_id` parameter is required, as it specifies which processing task the user is interested in. **Obtaining the task ID** The unique ID value for every processing task is returned in the response when the file is submitted for analysis, as the last part of the `task_url` field. For example, in `https://tiscale-worker01:8080/api/tiscale/v1/task/23`, the `task_id` value is 23. The task ID can also be obtained from the **List processing tasks** endpoint. ### Request format #### Request parameters | Parameter Name | Description | Type | | -------------- | ------------------------------------------------------------ | ----- | | `task_id` | Unique identification number assigned to each file processing task on the Worker instance. | path | | `report_type` | Applied report filter (see [Customizing Analysis Reports](../Usage/Analysis.md#customizing-analysis-reports)). If empty, a medium ("summary") report is returned. | query | | `view` | Applied report transformation. | query | | `top_container_only` | If set to `true`, the report will include metadata only for the top-level container. Reports for unpacked (child) files will be omitted. | query | | `malicious_only` | If set to `true`, the report will include only files and objects marked as malicious or suspicious. Benign content and clean children will be excluded from the report. | query | #### Deprecated and experimental parameters These parameters are maintained for backward compatibility or are experimental. - `include_fields` *experimental | query* Instead of creating a *report type* as a configuration file, you can pass the fields you want to include as a query parameter. The format is: `?include_fields=["metadata.media.image.exif","classification"]`. You can access nested fields using the dot (`.`) operator. All required fields should be passed as an array of strings in a **single query parameter**. - `exclude_fields` *experimental | query* Instead of creating a report type as a configuration file, you can pass the fields you want to exclude as a query parameter. The format is: `?exclude_fields=["metadata.media.image.exif","classification"]`. You can access nested fields using the dot (`.`) operator. All required fields should be passed as an array of strings in a **single query parameter**. - `full` *deprecated | query* Specifies whether the full (value 1; equals `true`) or summary (value 0; equals `false`) report should be returned. If this parameter is not provided in the request, and the `view` parameter is, the built-in summary transformation will be applied to the analysis report view. To avoid this transformation, the `full` parameter should be set to `true` if the `view` parameter is set to `extended`. - `v13` *deprecated | query* Specifies whether the report should be returned in the format used on Worker 1.3 (value 1, equals `true`) or in an improved format (value 0, equals `false`). Reports in version 1.3 represented enumerated values in their internal numeric value instead of as descriptive string values. For example, metadata for application library types was returned as numerical values 0 through 13 in version 1.3 instead of labels "ad", "social", "graphical" in the current version. This parameter should be used only for backward compatibility purposes because it does not take advantage of caching, which may slow down the API requests. **Request examples** ```bash curl 'https://tiscale-worker-01/api/tiscale/v1/task/7?report_type=large&view=flat' -H 'Authorization: Token exampletoken' ``` ```python url = "https://tiscale-worker-01/api/tiscale/v1/task/7" query_params = {"report_type": "large", "view": "flat"} auth = {'Authorization': 'Token exampletoken'} response = requests.get(url, headers=auth, params=query_params) print(response.text) ``` ### Response format The response returns information about the requested file processing task, and if the task has completed successfully, includes the analysis report. | Field Name | Description | |-----------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `submitted` | Unix timestamp indicating the time when the file was submitted for analysis. | | `task_id` | Unique identification number assigned to each file processing task on the Worker instance. This task ID can be used in requests to other Worker API endpoints; for example, to delete a task from Worker. To ensure backward compatibility with previous Worker versions, the task ID values are currently integers instead of strings. | | `processed` | Unix timestamp indicating the time when the file was processed by the Worker. If the file has not been processed yet, this field returns a null value until both the report and the file link are ready. | | `worker_ip` | List of IP addresses associated with the Worker that processed the file. | | `worker_address` | List of DNS records associated with the Worker that processed the file. | | `worker_hostname` | Hostname of the Worker that processed the file. | | `direct_sender` | IP address of the machine that sent the file to Spectra Detect. This is the immediate sender, the last hop in the forwarding chain (if the request was forwarded). | | `forwarded_for` | List of IP addresses that were part of the forwarding chain for that request, including the original sender. | | `tc_report` | Results of the static file analysis performed by the engine embedded in the Worker. If the report is already available, but the file has not yet been saved to S3, this field will return a null value. | | `file_link` | Returned only if the S3 integration is configured on the Worker, and if saving samples to S3 is enabled. When this field is present in the response, it includes a link to the sample stored in an S3 bucket. If Spectra Analyze integration and S3 are enabled on the Worker, the file link will point to the Spectra Analyze instance connected to the Worker. If report splitting is enabled, this link will point to the parent sample. | | `custom_data` | Returned only if the user submitted an optional `custom_data` parameter in the file upload request. This parameter can be passed to the `upload` endpoint to save arbitrary, user-defined data in the analysis report. The data must be submitted as a JSON-encoded payload, and it is returned in the report in its original, unchanged form. | | `deep_cloud_analysis` | Returned only if the Deep Cloud Analysis is enabled. When this field is present in the response, it includes the status of the Deep Cloud Analysis and the number of submitted (`submitted_av_scans`) and completed (`completed_av_scans`) AV scans. Available statuses of the Deep Cloud Analysis: `PREPROCESSING` - The file is still being unpacked (if the `Scan unpacked children` option is enabled), or the upload or re-scan of samples for that container is still in progress on Spectra Intelligence. `WAITING_FOR_AV_RESULTS` - Awaiting AV results, as not all AV scans are complete and the `Wait for AV scan` option is enabled for the analysis. `TIMED_OUT` - The wait timeout for AV scans has expired. Processing/reporting continues without all AV results being available. `FINISHED` - All AV scans have completed successfully. `NO_AV_ANALYSIS` - No AV analysis was triggered for this sample. For example, all samples may have been filtered out due to the `Rescan Interval` setting. `NO_WAIT` - Upload or re-scan was initiated on Spectra Intelligence, but the report will not wait for AV results to be included. `SUBSCRIBE_ONLY` - Deep Cloud Analysis is disabled, but the Cloud Classification Change feature is active. | ## Delete a processing task ``` DELETE /api/tiscale/v1/task/{task_id} ``` ### Description Deletes a processing task record from the system. All file processing results are automatically removed from the system 30 minutes after processing is completed. However, users can manually delete task records from the system at any time. When a task record is deleted, it is no longer listed in the response of the *List processing tasks* endpoint. The `task_id` parameter is required in the request, as it specifies which task record should be deleted. To bulk-delete task records, use the *Delete multiple processing tasks* endpoint. ### Request format **Request parameters** | Parameter Name | Description | Required | | -------------- | ------------------------------------------------------------ | -------- | | `task_id` | Unique identification number assigned to each file processing task on the Worker instance. The unique ID value for every processing task is returned in the response when the file is submitted for analysis, as the last part of the `task_url`. For example, in `"task_url": "https://tiscale-worker01:8080/api/tiscale/v1/task/23"`, the `task_id` value is 23. The task ID can also be obtained from the [List processing tasks](#list-processing-tasks) endpoint. | Required | **Request example** ```bash curl -X DELETE https://tiscale-worker01:8080/api/tiscale/v1/task/999 -H 'Authorization: Token exampletoken' ``` ### Response format If a task was successfully removed from the system, the API returns the `200 OK` status code with an empty response body. ## Delete multiple processing tasks ``` DELETE /api/tiscale/v1/task?{age=seconds} ``` ### Description Deletes task records from the system based on the time when they were submitted. All file processing results are automatically removed from the system 30 minutes after processing is completed. However, users can manually delete task records from the system at any time. When task records are deleted, they are no longer listed in the response of the [List processing tasks](#list-processing-tasks) endpoint. The `age` parameter is required, and it sets the limit for task age, expressed in seconds. Task age is calculated as being the difference between the current system timestamp and the timestamp of the task submission. Timestamps are specified in the Unix time format, as the number of seconds since 1970-01-01 00:00:00. All task records older than the number of seconds specified in the request will be deleted. To delete all tasks, set the `age` parameter to zero. ### Request format **Request parameters** | Parameter Name | Description | Required | | -------------- | ------------------------------------------------------------ | -------- | | `age` | Specify the number of seconds to delete processing tasks based on their age. Task age is the difference between the current system time and the time when the file processing task was created. The value must be specified in seconds. | Required | **Request examples** Delete tasks older than 500 seconds: ```bash curl -X DELETE 'https://tiscale-worker01:8080/api/tiscale/v1/task?age=500' -H 'Authorization: Token exampletoken' ``` Delete all tasks on the system: ```bash curl -X DELETE 'https://tiscale-worker01:8080/api/tiscale/v1/task?age=0' -H 'Authorization: Token exampletoken' ``` ### Response format The endpoint returns an empty body with one of the status codes, depending on the success of the request. | Code | Description | |------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | 200 | The request has succeeded. | | 400 | The request could not be understood by the server due to malformed syntax or missing parameters. | | 401 | The request requires authorization from the user. | | 403 | Authorization has been provided in the request, but it is not valid. | | 404 | The server has not found anything matching the Request-URI. | | 500 | The server encountered an unexpected condition which prevented it from fulfilling the request. | | 503 | The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. In case the Spectra Intelligence upload limit quota was exceeded, the response will contain the "Paused - TiCloud quota exceeded" error message. To continue analyzing files, users can temporarily disable Spectra Intelligence, wait until the daily quota renews, or contact ReversingLabs support to increase their daily upload limit. When using the Quotas API endpoints on a Worker instance without a configured Spectra Intelligence account, the response code will be 503, and the error message "TiCloud is not configured." | ## Get the identifier for YARA rules currently on Worker ``` GET /api/tiscale/v1/yara ``` ### Description Retrieves an identifier for the current set of YARA rules on the Worker. If there are any changes to the set of YARA rules, the identifier will change. Therefore, this endpoint can be used to monitor changes to YARA rules by comparing the responses retrieved over multiple time intervals. Read more about [managing YARA rulesets on the Worker](../Admin/YARASync.md). ### Request format **Request example** ```bash curl https://tiscale-worker-01/api/tiscale/v1/yara -H 'Authorization: Token exampletoken' ``` ### Response format **Response example** ```json { "id": "790473eae77e007cff9b88bc37a44596fa6a495a" } ``` ## Worker - Get YARA rule synchronization status. ``` GET /api/tiscale/v2/sync-status ``` ### Description Retrieves the current YARA rule synchronization status. Use the optional `errorOnly` query parameter to filter the response to include only rulesets with errors. If there are any changes to the set of YARA rules, the identifiers will change. Therefore, this endpoint can be used to monitor changes to YARA rules. ### Request format **Request example** ```bash curl https://tiscale-worker-01/api/tiscale/v2/sync-status -H 'Authorization: Token exampletoken' ``` ### Response format **Response example** ```json { "revision": 0, "rulesets": [ { "revision": 0, "key": "string", "hash": "string", "errors": [] } ] } ``` Appliances utilize revision numbers and keys as unique identifiers during ruleset synchronization. ## Worker - Resynchronize all YARA rules ``` POST /api/tiscale/v1/pull/all ``` ### Description Resynchronizes all YARA rules, ensuring the latest versions are retrieved and applied. This endpoint updates all rulesets regardless of their current revision status. ### Request format **Request example** ```bash curl -X POST https://tiscale-worker-01/api/tiscale/v1/pull/all -H 'Authorization: Token exampletoken' ``` ### Response format **Response example** If the request was successfully sent, the response includes a `200 OK` status code. In case the YARA synchronization is disabled, or the request fails, the response is `417 Expectation Failed`. ## Get information on usage quotas and limits {#quotas-api} The following endpoints can be used to return quota usage and quota limits for the Spectra Intelligence account configured on the Worker, or at the company level based on the Spectra Intelligence username. Products are grouped into one object if they share the usage quota and access rights. This means that the same users and products can appear multiple times in the response. Workers can be configured to require an authorization token in every API request. In that case, the `Authorization` header with a token must be included in the request. ``` GET /api/tiscale/v1/quota ``` ``` GET /api/tiscale/v1/quota/company ``` **Request Example** ```bash curl https://{server}/api/tiscale/v1/quota -H 'Authorization: Token exampletoken' ``` **Response Example** ```json { "limits": [ { "limit": 10, "limit_type": "daily", "limit_exceeded": false, "products": [ "TCA-0319 YARA Retroactive Hunting", "TCA-0303 YARA Hunting" ], "users": [ "u/company/user" ] }, { "limit": 500, "limit_type": "count", "limit_exceeded": false, "products": [ "TCA-0319 YARA Retroactive Hunting", "TCA-0303 YARA Hunting" ], "users": [ "u/company/user" ] } ], "usages": { "daily": { "date": "2023-11-23", "usage_report": [ { "product": "TCA-0103 Historic Multi-AV Scan Records", "number_of_queries": 6453 }, { "product": "TCA-9999", "number_of_queries": 87 }, { "product": "TCAI-0001 RCA2", "number_of_queries": 6415 }, { "product": "TCAI-0011 Sample Submission Counter", "number_of_queries": 17, "used_bytes": 171871643 } ] }, "monthly": { "month": "2023-11", "usage_report": [ { "product": "TCA-0103 Historic Multi-AV Scan Records", "number_of_queries": 7183 }, { "product": "TCA-0104 File Analysis - Hash", "number_of_queries": 6 }, { "product": "TCA-0201 File Download", "number_of_queries": 6 }, { "product": "TCA-0206 Alert on Reputation and Metadata Changes", "number_of_queries": 12 }, { "product": "TCA-0303 (Matches Feed)", "number_of_queries": 32642 }, { "product": "TCA-0319 (Matches Feed)", "number_of_queries": 32638 }, { "product": "TCA-0403 URL Threat Intelligence", "number_of_queries": 5 }, { "product": "TCA-0407 Network Reputation API", "number_of_queries": 20 }, { "product": "TCA-9999", "number_of_queries": 6273 }, { "product": "TCAI-0001 RCA2", "number_of_queries": 7403 }, { "product": "TCAI-0011 Sample Submission Counter", "number_of_queries": 64, "used_bytes": 352578582 }, { "product": "TCAI-0402 URI Statistics (RCA2)", "number_of_queries": 5 }, { "product": "TCF-0110 Data Change Feed", "number_of_queries": 427016 } ] } } } ``` --- ## Spectra Detect API — Manager, Service, Metrics, and Usage Endpoints --- ## Spectra Detect Open Source Licenses — Worker, Hub, and Manager RPM/NPM Packages --- ## Spectra Detect Appendix — Reference Materials and Open Source Licenses --- ## Spectra Detect Troubleshooting — Workers, Queues, YARA Sync, and Updates # Troubleshooting This guide covers common issues with [Spectra Detect](./index.md) deployments and the steps to resolve them. --- ## Worker node not appearing in SDM dashboard **Symptom** A newly deployed or restarted Worker node does not appear in the Spectra Detect Manager (SDM) appliance list, or its status shows as disconnected. **Cause** - The Worker is not configured with the correct SDM address or port. - A firewall is blocking the communication channel between the Worker and SDM. - The Worker has not been authorized in SDM. - TLS certificate mismatch between the Worker and SDM. **Solution** 1. On the Worker appliance, verify that the SDM address is correctly configured under [Appliance Configuration](./Config/ApplianceConfiguration.md). 2. Test network connectivity from the Worker to the SDM: ```bash curl -k https://:/api/v1/health-check/ ``` If this fails, check firewall rules between the Worker and SDM host. The required ports are documented in the deployment prerequisites. 3. In the SDM interface, navigate to **Appliances** and check whether the Worker appears as pending authorization. Newly connected Workers must be explicitly authorized before they appear as active. See [Manager Settings](./Admin/ManagerSettings.md). 4. If TLS certificates are custom, verify that the Worker trusts the SDM certificate. See [Certificate Management](./Admin/CertificateManagement.md) for certificate configuration. 5. Check SDM logs for connection rejection messages to identify the specific failure. --- ## File analysis queue is backing up **Symptom** The [Dashboard](./Usage/Dashboard.md) shows the analysis queue growing continuously. Files are taking much longer than normal to receive a classification result. The queue count in the SDM overview is increasing or not decreasing. **Cause** - The cluster does not have enough Worker nodes or CPU resources to keep up with the current ingestion rate. - A subset of Worker nodes is unhealthy or offline, reducing effective throughput. - Very large or deeply nested archives are occupying Worker instances for extended periods. - The input connector (S3, folder watch, or API) is ingesting files faster than Workers can process them. **Solution** 1. Check the status of all Worker nodes in the SDM dashboard. Confirm that all expected Workers are online and healthy. 2. Review Worker CPU and memory usage. Sustained CPU at 100% indicates the Workers are capacity-constrained. 3. For K8s deployments, scale the Worker deployment horizontally by increasing the replica count: ```bash kubectl scale deployment spectra-detect-worker \ --replicas= -n ``` --- ## YARA rules not syncing to Workers **Symptom** YARA rules uploaded to an appliance or to SDM do not appear on Worker nodes. Samples that should match a YARA rule do not produce expected results. The SDM YARA management page shows rules as uploaded but Workers do not reflect the changes. **Cause** - The YARA sync service on a Worker node is stopped or misconfigured. - A network connectivity issue is preventing SDM from pushing rules to the Worker. - A syntax error in a newly uploaded YARA rule is causing the entire ruleset to be rejected by the Worker. **Solution** 1. Verify that YARA sync is enabled and configured correctly on the Worker. See [YARA Sync](./Admin/YARASync.md) for the required configuration parameters. 2. Check YARA sync logs on the Worker node for error messages: ```bash # For OVA/AMI deployments journalctl -u spectradetect-yarasync -n 100 # For K8s deployments kubectl logs -c yara-sync -n ``` 3. If the logs show rule validation errors, identify the specific rule causing the failure. Invalid rules are typically logged with the rule name and a parse error message: ``` ERROR: YARA rule parse failed: rule "my_rule" at line 14 - unexpected token ``` Remove or correct the invalid rule and re-upload the ruleset. 4. Test SDM-to-Worker connectivity on the YARA sync port. Consult [Certificate Management](./Admin/CertificateManagement.md) if TLS is involved. 5. After resolving connectivity or rule issues, trigger a manual YARA sync from the SDM interface or via the [Management API](./API/ManagementAPI.md). --- ## S3 connector not picking up files **Symptom** Files placed in the configured S3 bucket are not being submitted for analysis. The analysis queue remains empty even though new files are present in the bucket. **Cause** - The S3 connector credentials (AWS access key or IAM role) are incorrect or lack the required permissions. - The bucket name or prefix path in the connector configuration does not match the actual bucket. - The S3 connector service is stopped or has encountered an error during startup. - The bucket is in a different AWS region than expected. **Solution** 1. Review the S3 connector configuration under [Analysis Input](./Config/AnalysisInput.md). Confirm the bucket name, region, and prefix are correct. 2. Verify that the IAM role or access keys have the following minimum permissions on the bucket: - `s3:GetObject` - `s3:ListBucket` - `s3:DeleteObject` (if files should be removed after processing) ```bash # Test access from the connector host aws s3 ls s3://// --region ``` 3. Check the connector service logs for authentication or connectivity errors: ```bash # For OVA/AMI deployments journalctl -u spectradetect-connector -n 100 # For K8s deployments kubectl logs -n ``` 4. Confirm the connector service is running. Restart it if it has crashed: ```bash # For OVA/AMI deployments sudo systemctl restart spectradetect-connector ``` 5. For K8s deployments, note that S3 connector support is provided through the Hub component. Refer to your deployment documentation for Hub configuration details. --- ## SDM shows appliance as unreachable **Symptom** In the SDM overview, one or more connected appliances (Workers or Spectra Analyze instances) display an "Unreachable" status. Alerts may be generated for the affected appliances. **Cause** - The appliance is powered off or has lost network connectivity. - The SDM heartbeat check is failing due to a temporary network disruption. - The appliance's management interface has changed IP address. - A TLS certificate on the appliance has expired, causing the SDM connection to fail. **Solution** 1. Attempt to access the appliance web interface or SSH directly to confirm it is online: ```bash ssh admin@ ping -c 4 ``` 2. In the SDM interface, navigate to **Manager Settings** and verify the registered IP address or hostname for the appliance. Update it if the IP has changed. See [Manager Settings](./Admin/ManagerSettings.md). 3. Check whether the appliance TLS certificate has expired. Certificate issues are logged in the appliance system logs and also visible on the Certificate Management page. See [Certificate Management](./Admin/CertificateManagement.md) for certificate renewal steps. 4. If the appliance is accessible but SDM still shows it as unreachable, restart the SDM communication service on the appliance: ```bash sudo systemctl restart spectradetect-sdm-agent ``` 5. Review SDM logs for specific error messages related to the unreachable appliance to identify the root cause. --- ## Analysis results not appearing in dashboard **Symptom** Files are being submitted and Workers show activity (CPU usage, queue movement), but analysis results do not appear in the SDM [Dashboard](./Usage/Dashboard.md) or in the analysis results view. **Cause** - The results reporting service on the Worker is misconfigured and is not sending results back to the Hub or SDM. - A downstream results consumer (webhook, SIEM integration, or S3 output bucket) is misconfigured, causing result delivery to fail silently. - The SDM database is not receiving result records due to a connectivity issue between the Hub and SDM. **Solution** 1. Check Worker logs for result delivery errors: ```bash kubectl logs -n | grep -i "result\|output\|error" ``` 2. Verify the output configuration under [Analysis Input](./Config/AnalysisInput.md). Confirm that the output destination (Hub address, S3 bucket, or webhook URL) is correct and reachable. 3. Test connectivity from the Worker to the Hub or SDM results endpoint. 4. For notification-based integrations (email alerts, webhooks), check the notification configuration. See [Notifications](./Usage/Notifications.md) for configuration options. 5. If results are being generated locally but not forwarded, check for disk space issues on the Worker node that might be causing result queuing to back up locally rather than forwarding to SDM. --- ## Update fails or gets stuck **Symptom** A software update initiated from the SDM **Updating** page (or via the update CLI) does not complete. The update status page shows the process as "In Progress" for an extended period, or the update fails with an error message. **Cause** - Network connectivity between the appliance and the ReversingLabs update server is disrupted during the download. - The appliance does not have sufficient disk space to stage the update package. - An SDM-managed appliance was rebooted or lost connectivity to SDM mid-update. - An incompatible update sequence (for example, skipping a required intermediate version). **Solution** 1. Navigate to the [Updating](./Admin/Updating.md) page in SDM and review the update log for specific error messages. 2. Check available disk space on the appliance before retrying: ```bash df -h ``` If disk space is insufficient, free space by purging old analysis results or logs before reattempting the update. 3. Test network connectivity to the ReversingLabs update infrastructure from the appliance: ```bash curl -I https://updates.reversinglabs.com ``` 4. Do not reboot or restart the appliance while an update is in progress unless instructed to do so in the error message. 5. If the update process is hung (no log activity for more than 30 minutes), contact [ReversingLabs Support](mailto:support@reversinglabs.com) before attempting to cancel or retry the update, as incomplete updates may leave the system in a partially upgraded state. 6. For K8s deployments, follow the upgrade procedure in your deployment documentation rather than using the SDM update mechanism. --- ## High memory usage on Worker nodes **Symptom** Kubernetes Worker pods are being OOM-killed, or node memory usage is consistently above 90%. The SDM dashboard or cluster monitoring (Prometheus, CloudWatch) shows frequent memory pressure on Worker nodes. **Cause** - The number of concurrent analyses (`concurrency-limit`) is set too high relative to the available memory per Worker pod. - Large files or archives with high decompression ratios are consuming more memory than typical workloads. - Memory limits set in the Helm chart are too low for the configured number of Spectra Core instances. **Solution** 1. Check if OOM events are occurring: ```bash kubectl describe pod -n | grep -i "OOMKilled\|Reason" dmesg | grep -i "oom\|killed" ``` 2. Review the Worker's Helm chart values and reduce the concurrency limit or the number of Spectra Core instances (`number-of-regular-cores`) to reduce peak memory usage. 3. Increase the memory limits and requests for Worker pods in the Helm values if available node memory allows it. Refer to your deployment's Helm values reference documentation. 4. Consider configuring a dedicated large-file Worker group with higher memory limits and a separate concurrency limit, leaving the regular pool for smaller files. Refer to your deployment documentation for Worker group customization options. 5. Review the [platform requirements](/General/DeploymentAndIntegration/PlatformRequirements) to confirm that the cluster nodes meet the recommended memory specifications for the configured number of Spectra Core instances. --- ## Cluster certificate errors **Symptom** Connections between cluster components fail with TLS errors. SDM reports appliances as unreachable. Browser access to the SDM web interface shows a certificate warning. Logs contain messages such as: ``` TLS handshake failed: x509: certificate has expired or is not yet valid TLS handshake failed: x509: certificate signed by unknown authority ``` **Cause** - Internal cluster certificates have expired and have not been renewed. - A custom CA certificate used to sign internal certificates is not trusted by all components. - The system clock on one or more nodes is incorrect, causing certificate validity window checks to fail. **Solution** 1. Identify which certificate is causing the failure by examining the TLS error in detail: ```bash openssl s_client -connect : -showcerts 2>&1 | openssl x509 -noout -dates ``` 2. Check certificate expiration across cluster components. See [Certificate Management](./Admin/CertificateManagement.md) for the full list of certificates used and the renewal procedure. 3. Renew expired certificates following the procedure documented in [Certificate Management](./Admin/CertificateManagement.md). After renewal, restart the affected services. 4. Verify that the system clock is synchronized on all nodes: ```bash timedatectl status chronyc tracking ``` If clock drift is detected, synchronize with NTP and verify that `ntpd` or `chronyd` is running and configured. 5. If an "unknown authority" error is present, ensure that the custom CA certificate is distributed to and trusted by all cluster components and the SDM host. See [Certificate Management](./Admin/CertificateManagement.md) for CA distribution steps. --- ## HAProxy service fails after upgrade to 6.0.0 **Symptom** After upgrading Spectra Detect Hub to version 6.0.0, the HAProxy service fails to start. **Solution** 1. Contact [ReversingLabs Support](mailto:support@reversinglabs.com) to obtain the `tiscale-hub-selinux-0.0.11-1.el8.noarch.rpm` package, then copy it to the Hub. 2. Reinstall the package: ```bash sudo dnf reinstall tiscale-hub-selinux-0.0.11-1.el8.noarch.rpm ``` 3. Reboot the Hub: ```bash sudo reboot ``` --- ## Analysis Timeout Issues File analysis timeouts can occur when processing complex or large files that require extensive analysis time. Understanding the causes and solutions helps ensure successful file processing. ## Common Causes Analysis timeouts typically happen due to: - **Large file sizes** - Files approaching or exceeding the size limits for your appliance tier - **Deep nesting** - Archives containing multiple layers of compressed files - **Extensive unpacking** - Files that trigger recursive decompression operations - **Complex file structures** - Files with intricate internal structures requiring detailed parsing - **Resource constraints** - Insufficient RAM or CPU allocation for the analysis workload ## Configuration Options ### Spectra Analyze The analysis timeout can be adjusted in the appliance configuration: 1. Navigate to **Administration > Configuration** 2. Locate the analysis timeout setting 3. Increase the timeout value based on your file processing requirements 4. Save the configuration changes ### File Inspection Engine Use the `--analysis-timeout` flag to control the per-file time limit: ```bash rl-scan --analysis-timeout 300 /path/to/file ``` The timeout value is specified in seconds. ## Troubleshooting Steps If analysis timeouts persist: 1. **Increase allocated resources** - Ensure the appliance or container has sufficient RAM (32 GB+ recommended) and CPU cores 2. **Check decompression ratio limits** - Verify that recursive unpacking isn't exceeding configured limits 3. **Review file characteristics** - Examine the file structure to identify potential issues 4. **Monitor system resources** - Check if the appliance is under heavy load from concurrent analyses 5. **Adjust timeout values** - Increase timeout settings for complex file processing workflows ## Related Topics - [Platform Requirements](/General/DeploymentAndIntegration/PlatformRequirements) - Hardware specifications for different appliance tiers - [How Spectra Core analysis works](/General/AnalysisAndClassification/SpectraCoreAnalysis) - Understanding the analysis process --- ## Antivirus Result Availability When a sample is uploaded or rescanned in Spectra Intelligence, it will usually get new antivirus results **within 30 minutes**. When a sample has new antivirus results, these will available in relevant APIs, for example [TCA-0104 File analysis](/SpectraIntelligence/API/FileThreatIntel/tca-0104/). --- ## Certificate Revocation ReversingLabs maintains a certificate revocation database that is updated with each [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis) release. Because the database is offline, some recently revoked certificates may not appear as revoked until the next update. Certificate Authority (CA) revocation alone is not sufficient to classify a sample as malicious. Most CAs backdate revocations to the certificate's issuance date, regardless of when or whether the certificate was abused. When additional context is available, ReversingLabs adjusts the revocation date to reflect the most appropriate point in time. If a certificate is whitelisted, this correction is not applied. ## Searching for Revoked Certificates You can find samples signed with revoked certificates using **Advanced Search** with the `tag:cert-revoked` keyword. Advanced Search is available both through the [Spectra Analyze user interface](/SpectraAnalyze/search-page/) and as the [TCA-0320 Advanced Search](/SpectraIntelligence/API/MalwareHunting/tca-0320/) API. --- ## File Classification and Risk Scoring — ReversingLabs # Classification File classification assigns a risk score (0-10) and threat verdict (malicious, suspicious, goodware, or unknown) to every analyzed file using ReversingLabs Spectra Core. The classification algorithm combines YARA rules, machine learning, heuristics, certificate validation, and file similarity matching to determine security status. YARA rules take precedence as the most authoritative signal, followed by other detection methods that contribute to the final verdict. The classification of a sample is based on a comprehensive assessment of its assigned risk factor, threat level, and trust factor; however, it can be manually or automatically overridden when necessary. Based on this evaluation, files are placed into one of the following buckets: - No threats found (unclassified) - Goodware/known - Suspicious - Malicious The classification process weighs signals from all available sources to arrive at the most accurate verdict. Some signals are considered more authoritative than others and take priority. For example, [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis) YARA rules always take precedence because they are written and curated by ReversingLabs analysts. These rules provide the highest degree of accuracy, as they target specific, named threats. This does not mean that other classification methods are less important. Similarity matching, heuristics, and machine learning still contribute valuable signals and may produce additional matches. In cases where multiple detections apply, YARA rules simply serve as the deciding factor for the final classification. ## Risk score A risk score is a value representing the trustworthiness or malicious severity of a sample. Risk score is expressed as a number from 0 to 10, with 0 indicating whitelisted samples from a reputable origin, and 10 indicating the most dangerous threats. At a glance: Files with no threats found don't get assigned a risk score and are therefore **unclassified**. Values from 0 to 5 are reserved for samples classified as **goodware/known**, and take into account the source and structural metadata of the file, among other things. Since goodware samples do not have threat names associated with them, they receive a description based on their risk score. Risk scores from 6 to 10 are reserved for **suspicious** and **malicious** samples, and express their severity. They are calculated by a ReversingLabs proprietary algorithm, and based on many factors such as file origin, threat type, how frequently it occurs in the wild, YARA rules, and more. Lesser threats like adware get a risk score of 6, while ransomware and trojans always get a risk score of 10. ### Malware type and risk score In cases where multiple threats are detected and there are no other factors (such as user overrides) involved, the final classification is always the one that presents the biggest threat. If they belong to the same risk score group, malware types are prioritized in this order: | Risk score | Malware types | |------------|---------------------------------------------------------------------------------------------------------------------| | 10 | EXPLOIT > BACKDOOR > RANSOMWARE > INFOSTEALER > KEYLOGGER > WORM > VIRUS > CERTIFICATE > PHISHING > FORMAT > TROJAN | | 9 | ROOTKIT > COINMINER > ROGUE > BROWSER | | 8 | DOWNLOADER > DROPPER > DIALER > NETWORK | | 7 | SPYWARE > HYPERLINK > SPAM > MALWARE | | 6 | ADWARE > HACKTOOL > PUA > PACKED | ## Threat level and trust factor The [risk score table](#risk-score) describes the relationship between the risk score, and the threat level and trust factor used by the [File Reputation API](/SpectraIntelligence/API/FileThreatIntel/tca-0101). The main difference is that the risk score maps all classifications onto one numerical scale (0-10), while the File Reputation API uses two different scales for different classifications. ### Nomenclature The following classifications are equivalent: | File Reputation API | Spectra Analyze | Spectra Detect Worker | | ------------------- | --------------- | ------------------------ | | known | goodware | 1 (in the Worker report) | In the Worker report, the [risk score](#risk-score) is called `rca_factor`. ## Deciding sample priority The [risk score table](#risk-score) highlights that the a sample's risk score and its classification don't have a perfect correlation. This means that a sample's risk score cannot be interpreted on its own, and that the primary criterion in deciding a sample's priority is its classification. Samples classified as suspicious can be a result of heuristics, or a possible early detection. A suspicious file may be declared malicious or known at a later time if new information is received that changes its threat profile, or if the user manually modifies its status. The system always considers a malicious sample with a risk score of 6 as a higher threat than a suspicious sample with a risk score of 10, meaning that samples classified as malicious always supersede suspicious samples, regardless of the calculated risk score. The reason for this is certainty - a malicious sample is decidedly malicious, while suspicious samples need more data to confirm the detected threat. It is a constant effort by ReversingLabs to reduce the number of suspicious samples. While a suspicious sample with a risk score of 10 does deserve user attention and shouldn't be ignored, a malicious sample with a risk score of 10 should be triaged as soon as possible. ## Acting on classification and risk tolerance When appliances or integrations surface data from ReversingLabs Threat Intelligence lookups, the **classification** and **rca_factor** fields are the primary signal to act upon. - Always evaluate the `classification` field first: it determines whether a sample is known goodware, suspicious, or malicious and therefore dictates priority. - Use `rca_factor` to adjust the risk aversion profile and suppress alerts for low-level threats when appropriate. Lower factors correspond to less severe detections. Additional fields worth reviewing: - `propagated`: `true` means one of the extracted child files caused the verdict, so you may need to investigate nested artifacts. - `scan_results`: contains all malicious detections found in the file; the worst detection is propagated as the final file verdict. - `result`: verbal verdict name. - `factor`: can usually be ignored because it is actionable only in conjunction with clean samples. - `propagated_source`: contains the extracted file hash that was found to be malicious and triggered propagation. ### Risk tolerance profiles 1. **Low risk tolerance**: the user wants to be alerted about any possible threat. In this profile, act on both **Suspicious** and **Malicious** verdicts, which generates the maximum number of matches. 2. **High risk tolerance**: the user only wants to be notified about highly impactful threats. In this profile, filter for `classification:Malicious` and `rca_factor >= 7` so only the most severe detections surface. For deployments using Spectra Analyze, the [Risk Tolerance Levels](/SpectraAnalyze/Administration/users-personalization/risk-tolerance-levels) guide explains how appliance sensitivity settings choose which analysis sources count toward classification. For more context on interpreting the classification structure and `rca_factor`, refer to the [Spectra Detect Analysis](/SpectraDetect/Usage/Analysis) documentation, which outlines the fields returned in Worker reports. ## Malware naming standard --- ## Handling False Positives # Handling False Positives A false positive occurs when a legitimate file is incorrectly classified as malicious. While ReversingLabs strives for high accuracy, false positives can occasionally happen due to the complexity of malware detection across hundreds of file formats and millions of samples. ## What You Can Do If you encounter a false positive, you have several options: ### 1. Local Classification Override On Spectra Analyze, you can immediately override the classification using the classification override feature: - Navigate to the file's Sample Details page - Use the classification override option to manually set the file as goodware - The override takes effect immediately on your appliance - All users on the same appliance will see the updated classification ### 2. Spectra Intelligence Reclassification Request Submit a reclassification request through Spectra Intelligence: - The override propagates across all appliances connected to the same Spectra Intelligence account - Other appliances in your organization will automatically receive the updated classification - This is the recommended approach for organization-wide corrections ### 3. Goodware Overrides Use Goodware Overrides to propagate trusted parent classifications to extracted child files: - If a trusted parent file (e.g., from Microsoft or another reputable vendor) contains files that trigger false positives - The parent's goodware classification can automatically override the child files - This is particularly useful for legitimate installers that may contain components flagged by heuristics ## How ReversingLabs Handles False Positive Reports If a customer reports a false positive (through Zendesk, or by contacting the Support team at support@reversinglabs.com), the first thing we do is re-scan the sample to make sure that the results are up-to-date. If the results are still malicious, our Threat Analysis team will: 1. Conduct our own research of the software and the vendor 2. Contact the AV scanners and notify them of the issue 3. Change the classification in our system (we do not wait for AVs to correct the issue) --- If the file is confirmed to be a false positive, we begin by analyzing why the incorrect classification occurred. Then we try to correct the result by making adjustments related to file relationships, certificates, AV product detection velocity (e.g. are detections being added or removed), we will re-scan and reanalyze samples, adjust/add sources and, if necessary, manually investigate the file. If these efforts do not yield a correct result, we have the ability to **manually override the classification** — but we only do so after thorough analysis confirms the file is benign. --- ## ReversingLabs malware naming standard The ReversingLabs detection string consists of three main parts separated by dots. All parts of the string will always appear (all three parts are mandatory). ``` platform-subplatform.type.familyname ``` 1. The first part of the string indicates the **platform** targeted by the malware. This string is always one of the strings listed in the [Platform string](#platform-string) table. If the platform is Archive, Audio, ByteCode, Document, Image or Script, then it has a subplatform string. Platform and subplatform strings are divided by a hyphen (`-`). The lists of available strings for Archive, Audio, ByteCode, Document, Image and Script subplatforms can be found in their respective tables. 2. The second part of the detection string describes the **malware type**. Strings that appear as malware type descriptions are listed in the [Type string](#type-string) table. 3. The third and last part of the detection string represents the malware family name, i.e. the name given to a particular malware strain. Names "Agent", "Gen", "Heur", and other similar short generic names are not allowed. Names can't be shorter than three characters, and can't contain only numbers. Special characters (apart from `-`) must be avoided as well. The `-` character is only allowed in exploit (CVE/CAN) names (for example CVE-2012-0158). #### Examples If a trojan is designed for the Windows 32-bit platform and has the family name "Adams", its detection string will look like this: ``` Win32.Trojan.Adams ``` If some backdoor malware is a PHP script with the family name "Jones", the detection string will look like this: ``` Script-PHP.Backdoor.Jones ``` Some potentially unwanted application designed for Android that has the family name "Smith" will have the following detection string: ``` Android.PUA.Smith ``` Some examples of detections with invalid family names are: ``` Win32.Dropper.Agent ByteCode-MSIL.Keylogger.Heur Script-JS.Hacktool.Gen Android.Backdoor.12345 Document-PDF.Exploit.KO Android.Spyware.1a Android.Spyware.Not-a-CVE Win32.Trojan.Blue_Banana Win32.Ransomware.Hydra:Crypt Win32.Ransomware.HDD#Cryptor ``` #### Platform string The platform string indicates the operating system that the malware is designed for. The following table contains the available strings and the operating systems for which they are used. | String | Short description | | ----------- | ------------------------------------------------------------------------------------------ | | ABAP | SAP / R3 Advanced Business Application Programming environment | | Android | Applications for Android OS | | AOL | America Online environment | | Archive | Archives. See [Archive subplatforms](#archive-subplatforms) for more information. | | Audio | Audio. See [Audio subplatforms](#audio-subplatforms) for more information. | | BeOS | Executable content for Be Inc. operating system | | Boot | Boot, MBR | | Binary | Binary native type | | ByteCode | ByteCode, platform-independent. See [ByteCode subplatforms](#bytecode-subplatforms) for more information. | | Blackberry | Applications for Blackberry OS | | Console | Executables or applications for old consoles (e.g. Nintendo, Amiga, ...) | | Document | Documents. See [Document subplatforms](#document-subplatforms) for more information. | | DOS | DOS, Windows 16 bit based OS | | EPOC | Applications for EPOC mobile OS | | Email | Emails. See [Email subplatforms](#email-subplatforms) for more information. | | Firmware | BIOS, Embedded devices (mp3 players, ...) | | FreeBSD | Executable content for 32-bit and 64-bit FreeBSD platforms | | Image | Images. See [Image subplatforms](#image-subplatforms) for more information. | | iOS | Applications for Apple iOS (iPod, iPhone, iPad…) | | Linux | Executable content for 32 and 64-bit Linux operating systems | | MacOS | Executable content for Apple Mac OS, OS X | | Menuet | Executable content for Menuet OS | | Novell | Executable content for Novell OS | | OS2 | Executable content for IBM OS/2 | | Package | Software packages. See [Package subplatforms](#package-subplatforms) for more information. | | Palm | Applications for Palm mobile OS | | Script | Scripts. See [Script subplatforms](#script-subplatforms) for more information. | | Shortcut | Shortcuts | | Solaris | Executable content for Solaris OS | | SunOS | Executable content for SunOS platform | | Symbian | Applications for Symbian OS | | Text | Text native type | | Unix | Executable content for the UNIX platform | | Video | Videos | | WebAssembly | Binary format for executable code in Web pages | | Win32 | Executable content for 32-bit Windows OS's | | Win64 | Executable content for 64-bit Windows OS's | | WinCE | Executable content for Windows Embedded Compact OS | | WinPhone | Applications for Windows Phone | ##### Archive subplatforms | String | Short description | | ---------------------------------- | ------------------------------------------------------------ | | ACE | WinAce archives | | AR | AR archives | | ARJ | ARJ (Archived by Robert Jung) archives | | BZIP2 | Bzip2 archives | | CAB | Microsoft Cabinet archives | | GZIP | GNU Zip archives | | ISO | ISO image files | | JAR | JAR (Java ARchive) archives | | LZH | LZH archives | | RAR | RAR (Roshal Archive) archives | | 7ZIP | 7-Zip archives | | SZDD | Microsoft SZDD archives | | TAR | Tar (tarball) archives | | XAR | XAR (eXtensible ARchive) archives | | ZIP | ZIP archives | | ZOO | ZOO archives | | *Other Archive identification* | All other valid [Spectra Core](/General/AnalysisAndClassification/SpectraCoreAnalysis) identifications of Archive type | ##### Audio subplatforms | String | Short description | | -------------------------------- | ---------------------------------------------------------- | | WAV | Wave Audio File Format | | *Other Audio identification* | All other valid Spectra Core identifications of Audio type | ##### ByteCode subplatforms | String | Short description | | ------ | ----------------- | | JAVA | Java bytecode | | MSIL | MSIL bytecode | | SWF | Adobe Flash | ##### Document subplatforms | String | Short description | | ----------------------------------- | ------------------------------------------------------------ | | Access | Microsoft Office Access | | CHM | Compiled HTML | | Cookie | Cookie files | | Excel | Microsoft Office Excel | | HTML | HTML documents | | Multimedia | Multimedia containers that aren't covered by other platforms (e.g. ASF) | | Office | File that affects multiple Office components | | OLE | Microsoft Object Linking and Embedding | | PDF | PDF documents | | PowerPoint | Microsoft Office PowerPoint | | Project | Microsoft Office Project | | Publisher | Microsoft Office Publisher | | RTF | RTF documents | | Visio | Microsoft Office Visio | | XML | XML and XML metafiles (ASX) | | Word | Microsoft Office Word | | *Other Document identification* | All other valid Spectra Core identifications of Document type | ##### Email subplatforms | String | Short description | | ------ | ------------------------------------- | | MIME | Multipurpose Internet Mail Extensions | | MSG | Outlook MSG file format | ##### Image subplatforms | String | Short description | | -------------------------------- | ------------------------------------------------------------ | | ANI | File format used for animated mouse cursors on Microsoft Windows | | BMP | Bitmap images | | EMF | Enhanced Metafile images | | EPS | Adobe Encapsulated PostScript images | | GIF | Graphics Interchange Format | | JPEG | JPEG images | | OTF | OpenType Font | | PNG | Portable Network Graphics | | TIFF | Tagged Image File Format | | TTF | Apple TrueType Font | | WMF | Windows Metafile images | | *Other Image identification* | All other valid Spectra Core identifications of Image type | ##### Package subplatforms | String | Short description | | ---------------------------------- | ------------------------------------------------------------ | | NuGet | NuGet packages | | DEB | Debian Linux DEB packages | | RPM | Linux RPM packages | | WindowStorePackage | Packages for distributing and installing Windows apps | | *Other Package identification* | All other valid Spectra Core identifications of Package type | ##### Script subplatforms | String | Short description | | --------------------------------- | ------------------------------------------------------------ | | ActiveX | ActiveX scripts | | AppleScript | AppleScript scripts | | ASP | ASP scripts | | AutoIt | AutoIt scripts (Windows) | | AutoLISP | AutoCAD LISP scripts | | BAT | Batch scripts | | CGI | CGI scripts | | CorelDraw | CorelDraw scripts | | Ferite | Ferite scripts | | INF | INF Script, Windows installer scripts | | INI | INI configuration file | | IRC | IRC, mIRC, pIRC/Pirch Script | | JS | Javascript, JScript | | KiXtart | KiXtart scripts | | Logo | Logo scripts | | Lua | Lua scripts | | Macro | Macro (e.g. VBA, AmiPro macros, Lotus123 macros) | | Makefile | Makefile configuration | | Matlab | Matlab scripts | | Perl | Perl scripts | | PHP | PHP scripts | | PowerShell | PowerShell scripts, Monad (MSH) | | Python | Python scripts | | Registry | Windows Registry scripts | | Ruby | Ruby scripts | | Shell | Shell scripts | | Shockwave | Shockwave scripts | | SQL | SQL scripts | | SubtitleWorkshop | SubtitleWorkshop scripts | | WinHelp | WinHelp Script | | WScript | Windows Scripting Host related scripts (can be VBScript, JScript, …) | | *Other Script identification* | All other valid Spectra Core identifications of Script type | #### Type string This string is used to describe the general type of malware. The following table contains the available strings and describes what each malware type is capable of. For a catalog of common software weaknesses that enable malware, see [CWE](https://cwe.mitre.org/) maintained by MITRE. CISA maintains advisories on actively exploited vulnerabilities at [cisa.gov/known-exploited-vulnerabilities](https://www.cisa.gov/known-exploited-vulnerabilities). | String | Description | | ----------- | ------------------------------------------------------------ | | Adware | Presents unwanted advertisements | | Backdoor | Bypasses device security and allows remote access | | Browser | Browser helper objects, toolbars, and malicious extensions | | Certificate | Classification derived from certificate data | | Coinminer | Uses system resources for cryptocurrency mining without the user's permission | | Dialer | Applications used for war-dialing and calling premium numbers | | Downloader | Downloads other malware or components | | Dropper | Drops malicious artifacts including other malware | | Exploit | Exploits for various vulnerabilities, CVE/CAN entries | | Format | Malformations of the file format. Classification derived from graylisting, validators on unpackers | | Hacktool | Software used in hacking attacks, that might also have a legitimate use | | Hyperlink | Classifications derived from extracted URLs | | Infostealer | Steals personal info, passwords, etc. | | Keylogger | Records keystrokes | | Malware | New and recently discovered malware not yet named by the research community | | Network | Networking utilities, such as tools for DoS, DDoS, etc. | | Packed | Packed applications (UPX, PECompact…) | | Phishing | Email messages (or documents) created with the aim of misleading the victim by disguising itself as a trustworthy entity into opening malicious links, disclosing personal information or opening malicious files. | | PUA | Potentially unwanted applications (hoax, joke, misleading...) | | Ransomware | Malware which encrypts files and demands money for decryption | | Rogue | Fraudulent AV installs and scareware | | Rootkit | Provides undetectable administrator access to a computer or a mobile device | | Spam | Other junk mail that does not unambiguously fall into the Phishing category, but contains unwanted or illegal content. | | Spyware | Collects personal information and spies on users | | Trojan | Allows remote access, hides in legit applications | | Virus | Self-replicating file/disk/USB infectors | | Worm | Self-propagating malware with exploit payloads | --- ## Risk score reference table --- ## How Spectra Core analysis works # How Spectra Core Analysis Works All ReversingLabs products are powered by [Spectra Core](https://www.reversinglabs.com/products/spectra-core) - the engine that analyzes every file and sample. The process of analyzing software involves several steps, and the final output are the analysis reports. To better understand the source and significance of the information contained in those reports, it's helpful to learn what Spectra Core does in the background of ReversingLabs products. This page provides an overview of the Spectra Core analysis process and explains what happens with files in each of the analysis steps. The following main steps have dedicated sections where they are described in detail: 1. [Identification](#1-identification) 2. [Unpacking](#2-unpacking) 3. [Validation](#3-validation) 4. [Metadata processing](#4-metadata-processing) 5. [Classification](#5-classification) ## Automated static analysis When you scan a file with Spectra Core, the engine automatically performs static analysis on the file and all files extracted from it. Automated static analysis is also referred to as **complex binary analysis**. This unique approach to software analysis decomposes files, collects their metadata, and classifies them in terms of the security risk they pose to end-users. Files are analyzed recursively, which means that every file extracted from the software package goes through the same analysis process like its container software package. As implemented in Spectra Core, automated static analysis does not require access to the source code (like SAST tools typically do). It can directly examine compiled software binaries to determine their structure, dependencies and behaviors. In addition to analyzing software binaries (which is the primary use-case), Spectra Core can analyze library code and source code for specific scripting languages. Another benefit of automated static analysis is that **files are not executed during the analysis process**. All available data is extracted even if the files are compressed, executable, or damaged - regardless of their target OS or platform. Because the analysis process does not execute any files, it can be completed in milliseconds and performed on very large files without significant performance penalties. All these features of automated static analysis give Spectra Core a unique advantage - it can analyze post-build artifacts and detect more novel, sophisticated software supply chain attacks than SCA tools are able to. SCA tools typically analyze package managers, manifest files, or source code repositories to find vulnerabilities. They are limited by the need for known signatures of open source dependencies that have to be cross-referenced against a vulnerability database. Being used in pre-build environments, SCA tools lack visibility into deep file structures and build process tampering evidence - insights that Spectra Core readily provides. ## The Spectra Core analysis process The process starts with the input file. The analysis engine performs several distinct steps on every object it extracts from the input file. The following diagram illustrates the flow that every object goes through. You can interact with the diagram to learn more about the process: - Select steps in the diagram to access their dedicated sections on this page ### 1. Identification Format identification is the initial step of the Spectra Core analysis process. To successfully perform the subsequent analysis steps, we first need to know the file format of every object we are analyzing. Specifically, this step analyzes the object structure to determine whether it's **binary** or **text**, and assigns the analyzed object a unique file format description. This description - file format identification - instructs the analysis engine on which rules and modules to use for further file processing. Two main approaches are used for format identification: - **Signatures** - created by ReversingLabs researchers to identify **binary** file formats based on their unique features. For example, Windows .exe files start with bytes "MZ", while PNG files will usually start with "‰PNG". Signatures describe expectations of what a file format should contain. Using heuristics, the analysis process checks whether those expectations align with the actual file structure. In addition to signatures, the analysis process also evaluates any relevant YARA rules (built into the engine as well as user-provided). If there are multiple matches, those from signatures take priority over YARA rule matches. - **Machine learning models** - created and trained by ReversingLabs researchers to identify **textual** file formats based on statistical text identification. The models are able to recognize basic text objects as scripting languages and distinguish software source code from other types of textual content. **Note: ✅ Completing the identification step** The results of the format identification step are: - File hashes - calculated by the analysis engine - File format descriptions - represented as File type.File subtype.Identification (for example, `Binary/Archive/ZIP`). If there are multiple versions of a file format, they can be identified through the additional `version` field. After the format has been identified, the file is either directed to the proper unpacking module according to its signature, or to the validation step. ### 2. Unpacking Unpacking, also referred to as **file decomposition**, is a step in the Spectra Core analysis process where the analyzed file is taken apart to extract all available components and metadata. During the unpacking process, the analysis engine eliminates obfuscation, encryption, compression, and any other protections that may have been applied to the file and its contents. The engine has built-in mechanisms to prevent infinite recursion, and supports configuring the decompression ratio and unpacking depth (how many layers of a file to extract). Different file formats require different unpacking approaches because of their structure and complexity. Because static analysis does not execute a file, it requires **unpackers** - specialized tools for parsing and unpacking individual file formats. ReversingLabs develops in-house static unpackers tailored to specific file formats, and Spectra Core relies on those unpackers during analysis. Generally speaking, goodware file formats are easier to unpack because their structure is known and well-defined, and file behavior can be observed from the format definition. File formats commonly used for malware are good at hiding code, which makes their unpacking more challenging. To create an unpacker for malware file formats, researchers have to identify each format and document its structure. The unpacker must be able to simulate file execution so that its code can be reconstructed and its behavior observed. Any obfuscation and protection artifacts must also be removed to allow extracting further objects. Information about the file behavior allows the unpacker - and consequently, the analysis process - to reveal the original software intent and to let users understand the true meaning of the code that was packed in that particular file format. The ability to unpack a file format makes it possible for the Spectra Core analysis engine to extract a wealth of metadata and critical information often not available from other tools. The collected metadata includes but is not limited to: format header details, strings (including secrets and URIs), function names, library dependencies, and file segments. Unpacking greatly increases the surface that can be analyzed and helps file classification by providing more metadata to look at. This makes it easier to confirm classification verdicts and increases the chance to catch every threat. **Note: ✅ COMPLETING THE UNPACKING STEP** After the file has been successfully unpacked, all collected metadata and the unpacked file content are passed to the validator assigned to the file format. The validator then performs integrity checks on the available data. ### 3. Validation Validation is a step in the Spectra Core analysis process where the **structure** and the **digital signatures** of the analyzed file are verified according to specific criteria for each file format. In the validation step, the previously identified file format is checked against its specification (the formal definition of the file format by its designer). In other words, the validation process looks for differences between the file format specification and its implementation. By doing this, we can gather additional information about the file format and detect anomalies in it. Any malformations that violate the file format specification are further examined to determine if they are capable of triggering potentially malicious behavior. Such malformations may be reported as known vulnerabilities. ReversingLabs uses these malformation patterns to create heuristics for potential future exploits and predictive vulnerability detection. Multiple validators may be used to verify a file format. They are called successively, first to last, or until one of them acknowledges that it recognizes and can handle the specific file format. If validation fails for one of them, the entire file is marked as invalid. Detected issues are reported as validation warnings or errors, depending on their severity. In addition to performing integrity checks of the file format structure, the validation step also verifies any digital certificates that have been used for code signing. Depending on its status, a certificate may influence the classification of files signed with it. The validation step assigns one of the following statuses to every detected certificate: - Valid certificate - Invalid certificate - Bad checksum - Bad signature - Malformed certificate - Self-signed certificate - Impersonation attempt - Expired certificate - Untrusted certificate - Revoked certificate **Note: ✅ COMPLETING THE VALIDATION STEP** After the file has been validated, all collected metadata is processed, evaluated, and transformed into actionable information that can be used to deliver the final file classification. ### 4. Metadata processing Metadata processing is a step in the Spectra Core analysis process where all previously collected metadata is translated into **human-readable**, **explainable information**. That information is used to produce or support the final file classification. Most of it is surfaced in Spectra Core analysis reports. In this step, metadata is converted into **capabilities** and **indicators**. They build up on the file format properties and platform-specific features of the analyzed file to describe software behavior and intent in more detail. The goal is to make it clearer what the analyzed code means and what each object is trying to do. #### Indicators Indicators can be described as behavior markers that are triggered when a specific pattern is found in the collected metadata or in the file content. An indicator may be triggered for multiple reasons. While some indicators can only be found in specific file formats, most are universal and therefore generally applicable. Indicators contribute to the final file classification, but not in an equal measure. Those deemed highly relevant are better at describing the detected malware type, while those with less relevant contributions help in solidifying the machine learning detection. #### Capabilities Based on the indicators triggered on a file, the analysis engine infers that the file exhibits a specific behavior, or that it is capable of performing specific actions. Similar software behaviors are grouped into broader categories - capabilities - according to the features they have in common. For example, a file can have the filesystem capability, which is a broad description that says the file can access the filesystem or perform filesystem operations, but doesn't describe which operation will actually take place. More fine-grained software behavior descriptions are derived from the indicators (e.g. "Accesses the httpd.conf file"). #### Tags The metadata processing step also assigns tags to files based on their properties such as certificate information, software behaviors, file contents, and many more. Some tags can only be applied to specific file types (for example, web browsers or mobile applications). Tags are visible in [Spectra Analyze](/SpectraAnalyze/system-and-user-tags) and can be queried through the [Spectra Intelligence Advanced Search (TCA-0320)](/SpectraIntelligence/API/MalwareHunting/tca-0320) API. In SAFE reports generated by Spectra Assure, tags appear for all unpacked files and for URIs in the Networking section, where they can be used for filtering. **Note: ✅ COMPLETING THE METADATA PROCESSING STEP** After the metadata has been fully processed, the file receives its classification status in the next step of the analysis. ### 5. Classification Classification is a step in the Spectra Core analysis process where the analysis engine produces a **verdict** on whether the analyzed file contains threats harmful to the end-user. Multiple technologies are used for file classification: - format identification - signatures (byte pattern matches) - file structure validation - extracted file hierarchy - file similarity (RHA1) - certificates - machine learning - heuristics (for scripts and fileless malware) - YARA rules included in the analysis engine They are shipped with the analysis engine and can be used offline, without connecting to any external sources. Their coverage varies based on threat and file format type. In other words, not all technologies can detect all threat types, and not all of them work on all file formats. Those default classification abilities of the Spectra Core platform can be extended with **threat intelligence from the ReversingLabs Cloud** to retrieve file reputation information, and with **custom YARA rules for user-assisted classification**. Some classification approaches are more specific than others, with signatures being the most specific. The final classification result relies on the information from all analysis steps, and it is a combination of all technologies applicable to the file format. It will always match one of the technologies even though they may have differing results between them. Because of differences in how malicious files and malware families behave, some files might end up classified as malicious by one technology, and still be considered goodware by others. This doesn’t negate or diminish the final classification. #### Explainable Machine Learning Spectra Core is the first and only solution on the market that relies on [Explainable Machine Learning (xAI)](https://www.reversinglabs.com/blog/machine-learning-for-humans) for threat detection. Explainable Machine Learning was launched by ReversingLabs in 2020 as a predictive threat detection method that can detect novel malware. It focuses on providing threat analysts with human-readable insights into machine learning-driven classifications. The goal of ReversingLabs Explainable Machine Learning is to go beyond the basic verdict of "goodware vs malware", and to help analysts understand **what type of threat was found**, **why it was detected**, and **what to do with it next**. To achieve that, the classification system combines: - **explainability** (by surfacing software behaviors in the form of indicators), - **relevance** (by ranking behaviors based on their contribution to the final verdict), - and **transparency** (by displaying why each software behavior was triggered). Using natural language to provide clear explanations for classification decisions helps security analysts understand how analyzed software behaves and what malware is capable of doing to the system. This transparency fosters trust, facilitates informed decision-making, and makes the logic behind machine learning classification verdicts easier to follow. Over the years, ReversingLabs threat analysts and researchers have carefully transformed raw code and metadata produced by static analysis into indicators - descriptions of software intent. Those indicators are used in training machine learning (ML) models to recognize if a file is malicious based on the described software functionality and behavior. Many of the threats in the training datasets are hand-picked by ReversingLabs experts and fully, correctly labeled so that ML models can learn what constitutes a specific threat type, and distinguish it from other threat types as well as from clean software. This allows ML models to proactively detect and describe threats - even brand new malware - without the need for additional training. When Spectra Core scans a file and extracts some indicators from it, ML models can match them against the indicators they have learned to recognize as typical for malware or a specific threat type. Some indicators are more meaningful in the context of a malware or threat type, so they contribute more to the classification. When the model decides that something is malicious, the decision can be verified through indicators and reasons why they were triggered. This makes the decision more transparent, relevant, and explainable in terms that are familiar to human analysts. ReversingLabs ML models are tailored to threat types to increase accuracy and [continuously improved](https://www.reversinglabs.com/blog/how-to-harden-ml-models-against-adversarial-attacks) to boost their resilience. All classification models can detect if a file is malicious or not. The PE (Portable Executable) malware classifier is also able to provide the information on the detected threat type. The exact threat type indicates higher confidence in the classification result, while threats that get assigned a generic threat type ("Malware") may point to new, emerging malware. The following ML models are used for malware classification: - PE malware classifier - detects if a file is malicious (that covers all the threat types) and if it is a specific malware type (one of **Backdoor**, **Downloader**, **Infostealer**, **Keylogger**, **PUA**, **Ransomware**, **Worm**) - Script classifiers - apply to `Text/