Skip to main content

Whitelisted files (new) (TCF-0501)

This service provides a continuous list of whitelisted samples in the Spectra Intelligence system.

The criteria for a file hash to be included in this feed: (1) file retrieved from trusted source, for example Microsoft; (2) file hash status has to be KNOWN (non-malicious or benign); and (3) at least 2 weeks have passed between the first scan and the latest scan.

If a file has one scan or the first and last scanning timestamps aren't at least 2 weeks apart, it will be queued for scanning after two weeks before it's put on the feed.

Every record in the feed includes hashes (MD5, SHA1, SHA256) and the platform field associated with the whitelisted sample. Users can extract samples related to a specific platform based on the value of the platform field.

The feed stores records for the last 365 days.

Continuous Whitelisted Feed Query​

This query returns a list of newly whitelisted samples since the requested time. The feed returns up to 1000 records and any surplus records sharing the same timestamp.

To fetch the next batch of records, use the timestamp from the response increased by 1.

If the requested timestamp is not within the last 365 days, the service will respond with the status code 400 Bad Request.

GET /api/feed/whitelisted/v1/query/{time_format}/{time_value}[?format=xml|json|tsv][&sample_available=false|true][&limit=N]

Request format​

  • time_format
    • Time format in which the date and time should be requested. It is possible to choose between utc and timestamp
    • Required
  • time_value
    • Accepts values in the format set by time_format. If the format is set to utc, the value should be expressed in the format YYYY-MM-DDThh:mm:ss. If the format is set to timestamp, the value should be expressed as the number of seconds since 1970-01-01 00:00:00
    • Required
  • format
    • An optional parameter that allows choosing the response format. Supported values are xml, json, tsv (Tab Separated Values, delimiter character t 0x09). The default is xml
    • Optional
  • sample_available
    • Optional parameter. If this parameter is set to true in the request, filtering will be applied and the response will contain only samples that are present in the ReversingLabs storage and available for download. When set to false, the query will return all samples, regardless of their download availability status. The default is false, meaning that if the parameter is not provided in the request, filtering is not applied.
    • Optional
  • limit
    • Specifies the number of records to return in the response. The maximum value is 1000. Note that the response may include a little more than the requested number of records to ensure that all the records with the same timestamp are returned.
    • Optional

Response format​

For the requested timestamp, the response contains a list of records. Every record in the list includes hashes (MD5, SHA1, SHA256), the record_on field associated with the time when the record entered the feed, and the platform field associated with the whitelisted sample. For samples with an undetermined platform, the value of the platform field will be Unknown.

The format of the last_timestamp value should match the requested time format.

An empty response is returned if no records for the requested timestamp are available.

    {
       "rl": {
          "whitelisted_feed": {
             "time_range": {
                "from": "YYYY-MM-DDTHH:MM:SS", 
                "to": "YYYY-MM-DDTHH:MM:SS"
                },
             "entries": [
                {
                   "sha1" : "sha1_value",
                   "md5" : "md5_value",
                   "sha256" : "sha256_value",
                   "platform" : "platform_value",
                   "record_on" : "record_on_value"
                },
                {...},
                …],
             "last_timestamp": "YYYY-MM-DDTHH:MM:SS_or_timestamp",
          }
       }
    }

PULL Query​

This query returns a list of whitelisted samples since a particular point in time. The starting point for this query is defined using the START query <start-query>.

If the user has not previously requested this query, nor has the START query been called, it will return records starting with the current timestamp. Every subsequent call will continue from the timestamp where the previous call ended. In case that the timestamp of the previous call is older than 365 days, the subsequent call will autocorrect this timestamp to the oldest available (i.e. current - 365 days), and corresponding records will be returned.

Unless the limit parameter is specified, the feed returns up to 1000 records and any surplus records sharing the same timestamp. That ensures all the records with the same timestamp will be included in the recordset. The limit parameter must not be greater than 1000.

This endpoint is built to be queried by single thread (single instance). Any concurrent requests will be blocked until the previous one is fulfilled.

GET /api/feed/whitelisted/v1/query/pull[?format=xml|json|tsv][&limit=N][&sample_available=false|true]

Request format​

  • format
    • An optional parameter that allows choosing the response format. Supported values are xml, json, tsv (Tab Separated Values, delimiter character t 0x09). The default is xml
    • Optional
  • sample_available
    • Optional parameter. If this parameter is set to true in the request, filtering will be applied and the response will contain only samples that are present in the ReversingLabs storage and available for download. When set to false, the query will return all samples, regardless of their download availability status. The default is false, meaning that if the parameter is not provided in the request, filtering is not applied
    • Optional
  • limit
    • Specifies the number of records to return in the response. The maximum value is 1000. Note that the response may include a little more than the requested number of records to ensure that all the records with the same timestamp are returned.
    • Optional

Response format​

The response format is the same as in the Continuous Whitelisted Feed Query <first-response>.

START Query​

This query sets the starting timestamp for the previously described PULL query <pull-query>.

The starting timestamp must be within the last 365 days, otherwise the service will respond with the status code 400 Bad Request.

PUT /api/feed/whitelisted/v1/query/start/{time_format}/{time_value}

Request format​

  • time_format
    • Time format in which the date and time should be requested. It is possible to choose between utc and timestamp
    • Required
  • time_value
    • Accepts values in the format set by time_format. If the format is set to utc, the value should be expressed in the format YYYY-MM-DDThh:mm:ss. If the format is set to timestamp, the value should be expressed as the number of seconds since 1970-01-01 00:00:00
    • Required

Response format​

A successful query returns an HTTP 200 OK message with an empty response body.

Examples​

1) Retrieving all new whitelisted samples from 2017-03-26 10:33:20

The examples return responses in XML as the default format, since the response format parameter is not specified.

/api/feed/whitelisted/v1/query/timestamp/1490517200
/api/feed/whitelisted/v1/query/utc/2017-03-26T10:33:20

2) Retrieving all new whitelisted samples from 2017-03-26 10:33:20 that are available in the storage

The examples use the timestamp format in the request. The first example returns the response in XML (as the default, not specified in the request), and the second example returns JSON (as requested).

/api/feed/whitelisted/v1/query/timestamp/1490517200?sample_available=true
/api/feed/whitelisted/v1/query/timestamp/1490517200?sample_available=true&format=json

3) Retrieving all new whitelisted samples from 2017-03-26 10:33:20 in JSON and XML format

/api/feed/whitelisted/v1/query/timestamp/1490517200?format=json
/api/feed/whitelisted/v1/query/timestamp/1490517200?format=xml