AWS EKS Micro Deployment [PREVIEW]

Introduction

PREVIEW

This deployment and its interfaces are under active development and subject to change. Compatibility is not guaranteed across minor updates during the beta period. Scope: Non‑production use

This document describes how the Spectra Detect platform is deployed and operated on Kubernetes, providing a high-volume, high-speed file processing and analysis that seamlessly integrates into existing infrastructure and effectively scales with business needs.

The platform is packaged as container images and deployment is managed with Helm charts, which package Kubernetes manifests and configuration into versioned releases for consistent installs, upgrades, and rollbacks.

Configuration is externalized via ConfigMaps and Secrets so behavior can be adjusted without rebuilding images, and sensitive data is stored separately with controlled access.

Horizontal Pod Autoscaling may adjust replica counts based on metrics such as CPU utilization or queue size.

Requirements

• EKS version 1.34, Amazon Linux 2023 with cgroupsv2 (tested)
• Persistent Volume provisioner supporting ReadWriteMany (e.g. Amazon EFS CSI).
• Helm 3 or above

Operators and Tools

Keda (autoscaling - optional)

For Spectra Detect to autoscale Workers, Keda needs to be installed on the cluster. Keda can be deployed following the official Deploying Keda documentation. It is not required to have Keda installed to run Spectra Detect on K8s, but it is required to utilize Worker autoscaling features.

Prometheus Operator

All pods have metrics exposed in Prometheus format. Prometheus can be used by setting:

# Can be enabled with the following
worker:
  monitoring:
    # -- Enable/disable monitoring with Prometheus
    enabled: true
    # -- Use actual release name
    prometheusReleaseName: "${PROMETHEUS RELEASE NAME}"

RabbitMQ Broker

Spectra Detect Helm charts support using external RabbitMQ Brokers (like AmazonMQ), as well as deploying and using RabbitMQ cluster resources as part of a Detect deployment installed in the same namespace. Choose which option to use based on the business requirements.

• External RabbitMQ Broker (deployed and managed outside of Spectra Detect Helmcharts)

  External/existing RabbitMQ Broker needs to be set up as per the broker installation guides. As an example, please check the Amazon MQ instructions.

• RabbitMQ Operator (deployed and managed by Spectra Detect Helm charts)

  Cloud native brokers can be deployed and managed by Spectra Detect Helm charts. RabbitMQ Operator needs to be installed in the K8s cluster.

kubectl apply --wait -f \  https://github.com/rabbitmq/cluster-operator/releases/download/v2.6.0/cluster-operator.yml

Secret	Type	Description
<release_name>-rabbitmq-secret	required	Basic authentication secret which contains the RabbitMQ username and password. Secret is either created manually (rabbitmq chart) or already exists.
<release_name>-rabbitmq-secret-admin	optional	Basic authentication secret which contains the RabbitMQ Admin username and password. Secret is either created manually (rabbitmq chart) or already exists. If missing, credentials from <release_name>-rabbitmq-secret.

PostgreSQL Server

Spectra Detect Helm charts support using external PostgreSQL Clusters (like Amazon RDS), as well as deploying and using PostgreSQL cluster resources as part of a Detect deployment.

• External PostgreSQL Server (deployed and managed outside of Spectra Detect Helm charts)

  External/existing PostgreSQL server needs to be set up as per the PostgreSQL server guide. As an example, please check the Amazon RDS instructions

• CloudNativePG Operator (deployed and managed by Spectra Detect Helm charts)

  CloudNativePG Operator needs to be installed in the K8s cluster.

# PostgreSQL Operator - CloudNativePG (CNPG)
kubectl apply --wait -f \  https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/release-1.21/releases/cnpg-1.21.1.yaml

Secret (deployment with Detect chart)	Type	Description
<release_name>-postgres-secret	required	Basic authentication secret which contains the database username and password. Secret is either created manually (postgres chart) or already exists.

Storage

Persistent volume

The /scratch folder is implemented as a persistent volume. Multiple services have access to the volume:

• Cleanup
• Preprocessor
• Processor
• Postprocessor
• Receiver
• Rabbit (if deployed)
• Postgres (if deployed)
• Connector S3 (if deployed)

Since multiple services (pods) are accessing the volume, the access mode of that volume has to be ReadWriteMany. In AWS, it is recommended to use EFS storage since it supports the requested access for pods.

Amazon EFS Remote Storage

If you are running Kubernetes on Amazon EKS, you can use Amazon EFS storage for the shared storage. You will need to:

1. Install Amazon EFS CSI Driver on the cluster to use EFS
2. Create EFS file system via Amazon EFS console or command line
3. Set Throughput mode to Elastic or Provisioned for higher throughput levels
4. Add mount targets for the node's subnets
5. Create a storage class for Amazon EFS

Ephemeral volume

The /tc-scratch folder is used as an ephemeral volume for temporary processing files.

• Processor

The requested access mode is ReadWriteOnce, and any type of storage that supports that can be used (e.g. in AWS encrypted-gp2).

The storage class name is empty by default (value tcScratch.storageClassName). If not overridden, emptyDir will be used for storage.

Getting Started

ReversingLabs Spectra Detect Helm charts and container images are available at registry.reversinglabs.com. In order to connect to the registry, you need to use the ReversingLabs Spectra Intelligence account.

helm registry login registry.reversinglabs.com -u "${RL_SPECTRA_INTELLIGENCE_USERNAME}"

If you want to see which versions of the charts are available in the registry, you can use a tool like Skopeo to log in to the registry and list the versions:

skopeo login registry.reversinglabs.com -u "${RL_SPECTRA_INTELLIGENCE_USERNAME}"

skopeo list-tags docker://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform

eg.
{
    "Repository": "registry.reversinglabs.com/detect/charts/detect-suite/detect-platform",
    "Tags": [
        "5.7.0-0.beta.3"
    ]
}

List of Detect Images

Image	Mandatory/Optional	Scaling * (see Appendix)
rl-detect-auth	Optional	N/A
rl-detect-receiver	Mandatory	CPU
rl-detect-preprocessor	Optional	QUEUE
rl-processor	Mandatory	QUEUE
rl-tclibs	Mandatory	CPU
rl-detect-postprocessor	Optional	QUEUE
rl-integration-s3	Optional	N/A
rl-report	Optional	CPU
rl-cloud-cache	Optional	N/A
rl-detect-utilities	Mandatory	N/A

Deploying Detect using Helm Charts

1. Create a namespace or use an existing one

   kubectl create namespace detect # Namespace name is arbitrary

2. Set up the Registry Secret to allow Kubernetes to pull container images.

   The Kubernetes secret rl-registry-key containing the user's Spectra Intelligence credentials needs to be created in the namespace where Detect will be installed.

   2.1. The secret can either be created via Detect Helm chart

   • registry.createRegistrySecret: needs to be set to true (default)
   • registry.authSecretName: value needs to be the Spectra Intelligence account username.
   • registry.authSecretPassword: value needs to be the Spectra Intelligence account password.

   2.2. Can be managed outside the Helm release

   • registry.createRegistrySecret: value should be set to false.

   You can create the secret manually by using the following command

   kubectl apply -n "detect" -f - &lt&ltEOF
   apiVersion: v1
   kind: Secret
   metadata:
     name: "rl-registry-key"
   type: kubernetes.io/dockerconfigjson
   data:
     .dockerconfigjson: $(echo -n '{"auths": {"registry.reversinglabs.com": {"auth": "'$(echo -n "${SPECTRA_INTELLIGENCE_USERNAME}:${SPECTRA_INTELLIGENCE_PASSWORD}" | base64)'"}}}' | base64 | tr -d '\n')
   EOF

3. Install a Spectra Detect Chart

   Choose any deploy name and run the following command.

   For more details regarding values, please reference the Appendix.

     helm install "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform \
   --version "${DETECT_HELM_CHART_VERSION}" --namespace "${NAMESPACE}" -f values.yaml

4. Configure Ingress Controller

   In order to be able to access Spectra Detect endpoints from outside the cluster, and in order for Worker pod to be able to connect to the Spectra Detect Manager, an Ingress Controller (like AWS ALB or Nginx Controller) must be configured on the K8s cluster. Follow the official installation guides for the controllers:

   • AWS Load Balancer Controller
   • Nginx Controller

   This example shows how to configure Ingress using AWS ALB Controller and how to use External DNS to automatically create DNS records in AWS Route53.

   Ingress values configuration example using AWS ALB Controller

   worker:
     ingress:
       annotations:
         alb.ingress.kubernetes.io/backend-protocol: HTTP
         alb.ingress.kubernetes.io/certificate-arn: <<AWS CERTIFICATE ARN>>
         alb.ingress.kubernetes.io/group.name: detect
         alb.ingress.kubernetes.io/healthcheck-interval-seconds: "15"
         alb.ingress.kubernetes.io/healthcheck-path: /readyz
         alb.ingress.kubernetes.io/healthcheck-port: "80"
         alb.ingress.kubernetes.io/healthcheck-protocol: HTTP
         alb.ingress.kubernetes.io/healthcheck-timeout-seconds: "5"
         alb.ingress.kubernetes.io/healthy-threshold-count: "2"
         alb.ingress.kubernetes.io/load-balancer-attributes: idle_timeout.timeout_seconds=600
         alb.ingress.kubernetes.io/manage-backend-security-group-rules: "true"
         alb.ingress.kubernetes.io/scheme: internal
         alb.ingress.kubernetes.io/security-groups: <<AWS_SECURITY_GROUPS>>
         alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-FS-1-2-Res-2020-10
         alb.ingress.kubernetes.io/success-codes: 200,301,302,404
         alb.ingress.kubernetes.io/target-group-attributes: stickiness.enabled=true,stickiness.lb_cookie.duration_seconds=1200
         alb.ingress.kubernetes.io/target-type: ip
         alb.ingress.kubernetes.io/unhealthy-threshold-count: "2"
         external-dns.alpha.kubernetes.io/hostname: detect-platform.example.com
         external-dns.alpha.kubernetes.io/ingress-hostname-source: annotation-only
       className: alb
       enabled: true
       host: detect-platform.example.com

Updating Detect using Helm Charts

Before running the upgrade, you can use the helm diff upgrade command to see the changes that will occur in the Kubernetes manifest files. Helm Diff Plugin must be installed to utilize the diff feature. You can install Helm Diff using the following command:

# Install plugin
helm plugin install https://github.com/databus23/helm-diff

# Run diff command
helm diff upgrade "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform \
  --version "${DETECT_CHART_VERSION}" \
  --namespace "${NAMESPACE}" \
  -f values.yaml

# Check the Helm chart readme beforehand if you want
helm show readme oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform --version "${DETECT_CHART_VERSION}"

# Run upgrade
helm upgrade "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform \
  --version "${DETECT_CHART_VERSION}" \
  --namespace "${NAMESPACE}" \
  -f values.yaml

Uninstalling Detect

helm uninstall detect -n "${NAMESPACE}"

Authentication

Authentication is achieved by leveraging Authentication based on SubRequest Result.

This is natively supported by the Nginx Ingress Controller. In case a different ingress controller is used (e.g. ALB on AWS), additional Nginx Reverse Proxy is deployed in order to support the authentication mechanism.

Authentication can be configured in the following ways:

1. Using external authentication service by specifying the externalAuhtUrl in the configuration

   If an external authentication service is enabled, all header values from the incoming request will be forwarded to the external authentication service.

   The external authentication service needs to return following responses in order to support this authentication mechanism:

   • HTTP 200: authentication successful (ingress will forward the traffic to the backend service)
   • HTTP 401 or HTTP 403: authentication failed

2. Using a simple authentication service deployed in the cluster

   Authentication service supports a simple Token check based on API path.

   The token needs to be included in the "Authorization" header with the "Token" prefix/scheme.

   curl -H "Authorization: Token <defined token>" <other options> <URL>

   The tokens are configured as secrets with the following behavior:

Secret	Type	Description	Used in deployments (Pods)
<release_name>-secret-worker-api-token	Optional	Token secret containing a token that is used to protect all endpoints with /api/ prefix, e.g. file upload.	Auth
<release_name>-secret-worker-api-task-token	Optional	Token secret containing a token that is used to protect /api/tiscale/v1/task endpoints. If left empty, the mentioned API is protected by <release_name>-secret-worker-api-token.	Auth

The authentication service won’t be deployed in the cluster if externalAuhtUrl is defined:

# Example for enabling authentication
worker:
  configuration:
    authentication:
      enabled: true
      externalAuthUrl: ""

Appendix

Set Report Types

• Report types can be added with the --set-file option by providing the name of the report type (added as a new key to reportTypes) and path to the file in which the filter is defined in JSON format
• Report type name must match the one defined in the given file
• Report types can be deleted by defining the name of the report.type and setting it to the value "" instead of adding the filepath
• Limitation: max size of the report type file is 3MiB

Set Report Types Example

# Example of adding the 2 report types
helm upgrade -i "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform -f custom_values.yaml \
--set-file worker.reportTypes.some_report_type=<path-to-report-type>/some_report-type.json
--set-file worker.reportTypes.extendedNoTags=<path-to-report-type>/extendedNoTags.json


# Example of adding the new report type and removing the existing report type
helm upgrade -i "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform -f custom_values.yaml \
--set-file worker.reportTypes.some_report_type=<path-to-report-type>/some_report-type.json \
--set-file worker.reportTypes.extendedNoTags="" \


############### Example of the report type file content ###############
{
  "name": "exampleReportType",
  "fields": {
    "info" : {
      "statistics" : true,
      "unpacking" : true,
      "package" : true,
      "file" : true,
      "identification" : true
},
    "story": true,
    "tags" : true,
    "indicators" : true,
    "classification" : true,
    "relationships" : true,
    "metadata" : true
  }
}

Use Report Type Example

Uploading a report type using the method above only makes the report type available to the system. To actually use the custom report type, you must configure it on the appropriate egress integration. For example, to use a custom report type with S3 storage:

# Specify the report type that should be applied to the Worker analysis report before storing it. Report types are results of filtering the full report. 
worker:
  configuration:
    reportS3:
      reportType: "custom_report_type"

Set YARA Rules

• YARA rules can be added with the --set-file option by providing the name of the rule file (added as a new key to yaraRules) and path to the file in which the rule is defined
• The rule file name must follow camelCase format
• YARA rules can be deleted by defining the rule file name and setting it to the value "" instead of adding the filepath
• Limitation: max size of YARA ruleset file is 3MiB

Set YARA Rules Example

# Example of adding yara rules
helm upgrade -i "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-platform -f custom_values.yaml \
--set-file worker.yaraRules.rule1=<path-to-yara-rule>/someYaraRule.yara \
--set-file worker.yaraRules.rule2=<path-to-yara-rule>/other_yara_rule.yara


# Example of adding the new yara rule and removing the existing yara rule
helm upgrade -i "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform -f custom_values.yaml \
--set-file worker.yaraRules.rule1=<path-to-yara-rule>/someYaraRule.yara \
--set-file worker.yaraRules.rule2=""


############### Example of the yara rule file content ###############
rule ExampleRule : tc_detection malicious
{
    meta:
        tc_detection_type = "Adware"
        tc_detection_name = "EXAMPLEYARA"
        tc_detection_factor = 5
    strings:
        $1 = "example"
        $2 = "eeeeeee"
    condition:
        $1 or $2
}

Set Advanced Filter

• Advanced filters can be added with the --set-file option by providing the name of the filter (added as a new key to advancedFilters) and path to the file in which the filter is defined in YAML format. Filter name must match the one defined in the given file
• Advanced filters can be deleted by defining the name of the filter and setting it to the value "" instead of adding the filepath
• Limitation: max size of YARA ruleset file is 3MiB

# Example of adding Advance Filter
helm upgrade -i "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform -f custom_values.yaml \
--set-file worker.advancedFilters.filter1=/some_filter.yaml \
--set-file worker.advancedFilters.filter2=/other_filter.yaml

# Example of adding Advance Filter
helm upgrade -i "${RELEASE_NAME}" oci://registry.reversinglabs.com/detect/charts/detect-suite/detect-platform -f custom_values.yaml \
--set-file worker.advancedFilters.filter1=/some_filter.yaml \
--set-file worker.advancedFilters.filter2=""

Example of the filter file content:

name: some_filter
description: Custom filter for Spectra Analyze integration
scope: container
type: filter_in
condition:
 and:
  - range:
     info.file.size:
      gt: 50
      lt: 20000
  - one_of:
     classification.classification:
      - 3
      - 2

Configuration Reference

info

View the Configuration Reference for more information about the configuration options.

Scaling

Scaling of the services is done in one of the following ways:

• Scaling based on CPU usage
• Scaling based on the number of messages waiting in the queue

Scaling based on CPU usage

Scaling based on CPU usage is implemented in the following way:

• Users can provide triggerCPUValue which represents the percentage of the given CPU resources. Service will be scaled up when this threshold is reached and scaled down when CPU usage drops below the threshold.
• CPU resources are defined with resources.limits.cpu value, which represents the maximum CPU value that can be given to the pod
• Default values:
  • scaling disabled by default
  • when CPU usage reaches 75%, the scaling is triggered
  • scaling delay is 30 seconds (scaling is triggered when scaling condition is met for 30 seconds)
  • one pod in every 30 seconds is created
  • number of maximum replicas is 8
  • every 10 seconds the average CPU status is checked
• Services with CPU scaling:
  • receiver
  • report
  • tclibs

Scaling based on the number of messages waiting in the queue

Scaling based on the number of messages waiting in the queue is implemented in the following way:

• Scaling disabled by default
• User can provide targetInputQueueSize which represents the number of messages in the queue for which the scaling will start
• Unacknowledged messages are excluded from this calculation
• Each scaled service has a different queue that is observed and scaled on
• Default values:
  • when 10 or more messages are waiting in the queue for a longer period of time the scaling will be triggered
  • scaling delay is 15 seconds (scaling is triggered when scaling condition is met for 15 seconds)
  • one pod in every 30 seconds is created
  • number of maximum replicas is 8
  • every 10 seconds status of the queue is checked
• Services with Queue based scaling:
  • processor: number of messages in the tiscale.hagent_input is used for scaling
  • processor-retry: number of messages in the tiscale.hagent_retry is used for scaling
  • postprocessor: number of messages in the tiscale.hagent_result is used for scaling
  • preprocessor: number of messages in the tiscale.preprocessing is used for scaling
  • preprocessor-unpacker: number of messages in the tiscale.preprocessing_unpacker is used for scaling

Scaling configuration

Value	Description
enabled	enables/disables auto-scaling
maxReplicas	maximum number of replicas that can be deployed when scaling in enabled
minReplicas	minimum number of replicas that need to be deployed
pollingInterval	interval to check each trigger (in seconds)
scaleUp	configuration values for scaling up
stabilizationWindow	number of continuous seconds in which the scaling condition is met (when reached, scale up is started)
numberOfPods	number of pods that can be scaled in the defined period
period	interval in which the numberOfPods value is applied
scaleDown	configuration values for scaling down
stabilizationWindow	number of continuous seconds in which the scaling condition is not met (when reached, scale down is started)
CPU scaling
triggerCPUValue	CPU value (in percentage), which will cause scaling when reached; percentage is taken from the resource.limits.cpu value; limits have to be set up
Queue scaling
targetInputQueueSize	Number of waiting messages to trigger scaling on; unacknowledged messages are excluded