File and URL Submissions
Submitting Files for Analysis
Files can be submitted to the appliance:
- manually through the graphical user interface:
- as a direct file upload
- as a link to the file (which the appliance then downloads)
- using the Submissions API
- using a Connector, which pulls files from one or more configured sources
- by pivoting from a previously analyzed sample:
- from the Spectra Detect Manager dashboard
- from an S3 file link in a Spectra Detect Worker report
Manual uploads
A progress bar in the header indicates the upload status while files are uploading. Navigating away from the page or refreshing the browser tab during upload is not supported and will cancel the upload.
Uploaded samples will follow your current analysis settings, but you can make a one-time adjustment for each upload, such as sending the file to Spectra Intelligence for threat reputation info, or using one of the sandbox integrations.
Some security solutions opt to put suspicious and malicious files, such as email attachments, into password-protected archives before passing them on for further analysis, using the archive format as a secure means of transport. If you're submitting such a password-protected archive, you can also provide a one-time password here.
This feature expects the ZIP file to contain only one file and will, upon successful extraction, upload only the extracted file and discard the archive. Only CRC32 encrypted ZIP files are supported. AES encryption is not supported at this time.
This specific unpacking mechanism is triggered by providing a password. For general password-protected archive usage uploads (where multiple files can be extracted and processed), perform a regular upload with a preconfigured password list (Administration > Configuration > Password List).
File Processing
When a file is submitted to the appliance, it is processed with Spectra Core. The file becomes visible in the Local tab on the Search page. Detailed analysis results can be viewed in the Expanded Details section and on its Sample Details page.
The duration of the analysis depends on file sizes and file types, as well as the amount of files extracted during analysis, which are also analyzed separately.
After the file is processed with Spectra Core, users can optionally submit it to Spectra Intelligence and/or supported dynamic analysis services. This integration with Spectra Intelligence and dynamic analysis services must be configured by appliance administrators.
Depending on the appliance configuration, samples with supported file types can be automatically sent to dynamic analysis services after they are submitted to the appliance.
For CAPE and Joe Sandbox, previously analyzed files will not be automatically sent for analysis again if the Submit only distinct files option is configured. Administrators can configure this on the Administration ‣ Integrations page.
File Size Restrictions
- The maximum supported file size for upload on Spectra Analyze is 10 GB. This value can be configured in Administration > Configuration > General > File Size Limit.
- Files larger than 400 MB cannot be submitted for dynamic analysis (individual dynamic analysis integrations have even lower limits).
- YARA rulesets are not applied to extracted files larger than 700 MB.
New files cannot be submitted to Spectra Analyze if the disk space usage on the appliance exceeds the set value. If this happens, either manually remove old samples, ask the administrator to run the Backup and Purge action before continuing to submit new files, or increase the disk size in Administration > Configuration > Resource Usage Limits (see System Configuration for more information).
Pivoting from Spectra Detect
If Spectra Analyze is connected to Spectra Detect (either to individual Workers, or a cluster managed by Spectra Detect Manager), it can pull files from a preconfigured S3 bucket or directly from Spectra Detect Manager. Both of these options must first be configured on Spectra Detect.
The pivot link is present in the dashboard of Spectra Detect Manager, as well as in the Worker JSON report under file_link
. When you open the link, Spectra Analyze will pull the previously analyzed file from the preconfigured source and will reanalyze it.
If the version of Spectra Detect is 5.3 or higher, the imported file will also be tagged with the spectra_detect
tag.
Submitting URLs for Analysis
URLs can be manually uploaded to the Spectra Analyze from any page of the interface by clicking the Upload button on the header bar and selecting Submit URL from the menu, or via the Submissions API as part of an automated workflow.
In the URL submission dialog that opens, enter the full URL of a website including the protocol (https://www.example.org), or a full link to a single file (http://www.example.org/documents/reports/year-report.pdf). Supported protocols are HTTP and HTTPS.
- If the URL links to a single file, the file is downloaded to the appliance in its original format (for example: PDF, EXE, RTF…). The URL still has a dedicated Network Threat Intelligence page, and the file submission is related to it as its payload.
- If URL links to a website, the appliance uses a simple web crawler that retrieves the content up to 1 level from the submitted URL (including links to other domains). The scraped content is downloaded to the appliance in a single ZIP file.
Select the crawling method to be used to crawl the URL. For more information on these methods, refer to the Privacy of Submitted Files and URLs chapter.
Click OK to confirm URL submission, or Cancel to close the submission dialog. The submission cannot be confirmed if the URL is invalid.
The Search page displays the analysis results for files downloaded from the submitted URL. If any of the files in the scraped content are malicious or suspicious, the final verdict for the ZIP file - therefore, the URL - is malicious or suspicious.
The submission type indicator icon on the left side of the page helps distinguish between files downloaded to the appliance via a URL (the link icon) and files directly submitted to the appliance (the folder icon).
Analyzing Data from Submitted URLs
The duration of the analysis depends on the number of files downloaded from the submitted URL, their sizes and file types, as well as the amount of files extracted during analysis, which are also analyzed separately. The timeout for URL submissions is 45 minutes.
By default, files downloaded from submitted URLs are analyzed with the Spectra Core static analysis engine built into Spectra Analyze. Users can manually send those files for analysis to Spectra Intelligence and/or configured dynamic analysis services using the Reanalyze option. This integration with Spectra Intelligence and dynamic analysis services must be configured by appliance administrators.
All files and websites downloaded to the appliance via the URL submission dialog are automatically assigned the URL Download User Tag. This tag is visible in the Expanded Details and on the Sample Details page for every file and website. Clicking the tag opens the Tags page filtered to display all files with the URL Download tag. Users can then sort the files and perform bulk actions, such as reanalyzing them or adding them to alert subscriptions.
URL Submission Restrictions
The maximum allowed size of data to download from submitted URLs can be configured by the appliance administrator. By default, it is limited to 200 MB. This refers to all data downloaded from a URL, not just to the size of a single file. This value is configurable by appliance administrators in the Administration ‣ Configuration ‣ URL Analysis dialog. The maximum configurable value is 700 MB.
In addition to the URL submission size limit, submitting a URL using the Spectra Intelligence crawling method will also compare individual components of the submitted URL to the Maximum Fetch File Size value in Administration > Configuration > Spectra Intelligence. Any files going over this limit will be skipped. The maximum configurable value is 500 MB.
If the download request fails, the URL submission is marked as failed. Users can attempt to reanalyze the submission by selecting the Retry analysis option in the actions menu (☰). This option is available for individual submissions only (not for multiple submissions at once).
Privacy of Submitted Files and URLs
File Submissions
All files submitted to the appliance are accessible to all users with accounts on that Spectra Analyze instance.
While each submission is associated with a particular user (the one who submitted the file or URL), actual files on the local appliance system are not owned by any of the users in the traditional sense of file ownership. Therefore, all users on the Spectra Analyze instance can download, reanalyze, subscribe/unsubscribe, add tags, and manually change classification for any file uploaded by another user.
URL Submissions
Depending on which crawling method is selected, submitted URLs are treated differently.
- The
Local
crawling method will treat the URL as any other locally submitted file. The contents of the URL are crawled and downloaded directly. This method can be used without a Spectra Intelligence account. If Spectra Intelligence is configured and is using a proxy, the same proxy will be used to crawl the URLs when using this method. - The
Spectra Intelligence
crawling method is more reliable when working in restricted network conditions and ensures fewer failed URL analyses. However, all submitted URLs and downloaded files are treated as public, and will be visible and accessible to all Spectra Intelligence users. The prerequisite for this is a properly configured Spectra Intelligence account on the appliance.
The default crawling method can be configured in the URL Analysis section of the Configuration page, but users can freely change the crawling method on every URL submission. Next time the Submit URL window is opened, it will default to the last used crawling method.
Appliance administrators can delete files submitted by other users. Regular users can only delete their own submissions.
Spectra Intelligence
If the appliance is connected to Spectra Intelligence, all submissions can be:
- Manually uploaded to be analyzed with AV engines. This is done with the Reanalyze option.
- Automatically uploaded (Administration > Configuration > Spectra Intelligence > Automatic Upload to Spectra Intelligence). This is disabled by default.
Whether submitted files will be shared with other ReversingLabs customers depends on the role configured for the Spectra Intelligence account used by the appliance.
Spectra Intelligence accounts created to be used with Spectra Analyze appliances are always configured as private (non-shareable), meaning that other ReversingLabs customers may only be able to access analysis results for the files, but not retrieve their contents.
However, if those same files are uploaded to Spectra Intelligence as shareable from another source, they will cease to be treated as private. In that case, other ReversingLabs customers may be able to download the files, their metadata, and their analysis results through other ReversingLabs solutions (such as APIs and Feeds).
If Spectra Intelligence is not configured on Spectra Analyze, files are only preserved on the local appliance system and accessible only to users on that instance.
ReversingLabs Cloud Sandbox
Whether submitted files, PCAP files, dropped files, and memory string dumps will be shared with other ReversingLabs customers depends on the role configured for the Spectra Intelligence account used to upload files.
If the account is configured to upload all files as not shareable (private), other ReversingLabs customers will only be able to access analysis results, but not retrieve the actual contents of uploaded files, dropped files, PCAP files or memory string dumps. This is the default setting for Spectra Intelligence accounts created to be used on Spectra Analyze appliances.
If the account is configured to upload all files as shareable (not private), other ReversingLabs customers will be able to access analysis results, but also download the uploaded files, dropped files, PCAP files, and memory string dumps generated during file execution.