YARA Retroactive Hunting

The YARA Retroactive Hunting feature is an extension of the already powerful yara capabilities available on the Spectra Analyze appliance.

With standard YARA hunting, only new samples are compared against YARA rules, and new matches are reported as they happen. Retroactive Hunting allows users to scan through old samples to uncover YARA rule matches that would otherwise remain hidden.

Three types of YARA Retroactive Hunting are supported on Spectra Analyze, with one being available only if the appliance is in a Spectra Detect cluster:

• Cloud Retro Hunting, which applies to samples in the Spectra Intelligence cloud that were analyzed in the past 90 days. Archives and samples larger than 200 MB are excluded from the sample set, but samples extracted from those archives are not.
• Local Retro Hunting, which applies to all samples on the local Spectra Analyze instance regardless of their age and analysis date. Local Retro Hunts across large datasets are processing-intensive and may negatively impact the performance and stability of the appliance.
• Remote Storage Hunting, which applies to samples stored in S3 buckets configured on a Spectra Detect Hub group S3 connector service. Such retro hunts will be executed using the Worker appliances, and the results will be visible on the Manager dashboard. Links to remote storage retro hunts can be found in the YARA Retro Hunt pop-up dialog.

Local Retro Hunting

Important notes

1. Local Retro Hunting can be performed on all active rulesets on the appliance, including Spectra Core rulesets.
2. Local Retro Hunting cannot be performed on individual rulesets - it runs on all active rulesets at once.
3. To exclude a ruleset from Local Retro Hunting, disable it before running Local Retro.
4. Local Retro does not submit samples to dynamic analysis services (such as Cuckoo or FireEye) during a scan.

Starting a Local Retro Hunt

To start a Local Retro scan on all active rulesets on the appliance, click the Run Retro Hunt button at the top right of the YARA page. This opens a pop-up with the options to perform a retro hunt on local samples, or to initiate a retro hunt on remote storage.

tip

Performing retro YARA hunts on remote storage is available only if the appliance is in a Spectra Detect cluster comprising a Spectra Detect Manager, Hub and Worker with a configured S3 connector on the Hub group. Users can select the buckets from those added to the Hub group’s S3 connector and configure the desired file age and folder/prefix for the files to be included in the hunt. Such retro hunts will be executed using the Worker appliances, and the results will be visible on the Manager dashboard. Links to remote retro hunts can be found in the YARA Retro Hunt pop-up dialog.

To stop a retro scan, open the YARA Retro Hunt List menu (the clock icon) and click the stop button next to the retro hunt listing. Local Retro does not submit samples to dynamic analysis services (such as Cuckoo or FireEye) during a scan.

When the scan completes, the results will be available in the list of YARA matches for every active ruleset. The results for Local Retro scans cannot be cleared, and samples cannot be removed from the list of matches.

Stopping and Restarting a Local Retro Hunt

While a Local Retro scan is in progress, it can be stopped by clicking the stop button next to the retro hunt listing in the YARA Retro Hunt List dialog.

After a Local Retro scan is stopped or completed, it can be restarted at any point. However, the next scan behaves as a fresh start - it does not continue from where it previously stopped.

The YARA Retro Hunt List dialog contains detailed information about all previous Local Retro scans, including whether they were stopped or not.

Cloud Retro Hunting

Important notes

1. Cloud Retro Hunting can be performed only on rulesets that are saved in the Spectra Intelligence cloud, because the Cloud Retro sample set resides there. For a successful Cloud Retro scan, a ruleset needs to be able to access that sample set.
2. The Cloud Retro sample set includes samples analyzed in the Spectra Intelligence cloud over the last 90 days. Archives and samples larger than 200 MB are excluded, but samples extracted from those archives are not. The sample set is not static, and it changes daily.
3. Cloud Retro Hunting cannot be performed on Spectra Core rulesets.
4. The maximum number of Cloud Retro scans that can be executed depends on the license purchased by the customer.
5. The maximum amount of rules and rulesets that can be scheduled for Cloud Retro Hunting at a time depends on the license purchased by the customer.

Starting a Cloud Retro Hunt

To enable Cloud Retro Hunting for a ruleset, the selected ruleset must first be saved to Spectra Intelligence.

To do this, open the ruleset editor by clicking the Edit item in the ruleset action menu (☰). In the editor, select the Run ruleset continuously in Spectra Intelligence checkbox and click the Save or Save & Close button. Alternatively, open the ruleset's page and enable the Run Ruleset in Cloud switch.

The indicators on the YARA page should show that the ruleset is active in the cloud and ready for Cloud Retro. An indicator of synchronization status will be visible in the upper right part of the YARA page. The indicator displays the date and time of last synchronization, and allows skipping the synchronization until the next day.

The action menu for the ruleset should now have the Run Cloud Retro Hunt option. Selecting it will schedule the ruleset for a Cloud Retro Hunting scan. If the ruleset has been successfully validated, the retroactive scan will start shortly.

Alternatively, the appliance administrator can enable automatic Cloud Retro hunting for YARA rulesets saved to Spectra Intelligence in the Administration > Configuration > YARA Cloud Settings configuration dialog.

If this option is enabled, YARA rulesets that are synchronized with Spectra Intelligence will be automatically scheduled for a Cloud retro scan that will start after successful ruleset validation. This applies to new rulesets created on the appliance, and to existing rulesets that are edited and synchronized with Spectra Intelligence by selecting the “Run ruleset continuously in Spectra Intelligence” checkbox in the YARA ruleset editor.

If new samples are uploaded to the Spectra Analyze appliance while a Cloud Retro scan is in progress, they will be also matched against the ruleset that is being scanned.

During a Cloud Retro scan, the progress for the ruleset is displayed next to the status indicator icons on the YARA page.

When the Cloud Retro scanning for a ruleset is completed, the RETRO indicator on the YARA page will reflect that the scan was successful. Hovering over the indicator displays a tooltip with more information on the ruleset’s Cloud Retro status.

If the scan has completed successfully, the tooltip will show the date and time when the latest Cloud Retro hunt started and finished.

Stopping and Restarting a Cloud Retro Hunt

It is possible to stop a Cloud Retro scan if the user needs to modify the ruleset for accuracy, or because of performance concerns. While a Cloud Retro scan is in progress, click the triple bar button (☰) on the right side of the selected ruleset. Choose the Stop Retro Hunt item from the menu.

After a Cloud Retro scan is stopped or completed, it can be restarted at any point (as long as the ruleset remains saved to Spectra Intelligence).

However, restarting a Cloud Retro scan on a ruleset will remove the results from the previous Cloud Retro scan. The restarted scan behaves as a fresh start - it does not continue from where it previously stopped.

Managing YARA Retroactive Hunting Results

To view and manage the results of Retroactive Hunting scans, click a ruleset name to expand the list of matched samples.

Cloud Retro scan results will have their source displayed as a cloud icon. If the file is not available for download from Spectra Intelligence, a light gray cloud icon is displayed

The list of matches for Retroactive Hunting contains similar options as described in the Managing YARA Ruleset Matches section. Users can choose the ruleset version from the dropdown list below the ruleset name to compare scan results for each version. Every sample in the list of results can be selected and managed individually using the options in the action menu (☰). Clicking the file name opens the local or the Spectra Intelligence version of the Sample Details page for the selected file, depending on if the sample is locally available.

The Retroactive Hunting results can also be managed in bulk. Selecting one or more samples in the list of results activates the triple bar button (☰) on the right side of the Size column.

The button opens the bulk action menu with options for:

• downloading and analyzing selected samples on the Spectra Analyze appliance, if the samples are available for download (cloud and cloud-retro only),
• removing samples from the list of matches (cloud and cloud-retro only),
• reanalyzing samples (local and local-retro only),
• applying tags to samples (local and local-retro only),
• downloading samples to local storage,
• adding/removing samples in new or existing alert subscriptions,
• exporting data about selected samples as a CSV file.

Removing Samples from the Cloud Retro Results

Samples can be removed from the list of Cloud Retro results manually by selecting them and choosing the Remove Cloud Matches option from the action menu (☰). Using this method, samples can be removed one by one, or the user can select and remove multiple samples at once.

Additionally, it is possible to clear all Cloud Retro results for a selected YARA ruleset. To do this, select a ruleset and click the triple bar button (☰) on the right side of the ruleset row. In the action menu that opens, select Clear Retro Hunt.

When the Cloud Retro results for a ruleset are cleared, the icon for the RETRO indicator on the YARA page will change, showing that the results have been cleared. Users can then run a new Cloud Retro scan on the ruleset at any point.