# Reputation & Metadata Alerts API (tca-0206) — Spectra Intelligence

> Subscribe to up to 1 billion file and URL hashes in Spectra Intelligence via tca-0206 and receive Data Change Feed notifications when classification or metadata changes.

# Reputation and metadata alerts (TCA-0206)

The Reputation and Metadata Changes service allows users to subscribe to up to one billion sample and URL hashes, tracking changes in their status over a period of time. These subscriptions, valid for a duration of 1 year, enable users to monitor any changes to the subscribed samples or URLs. When changes occur, they are delivered through the [Data Change Feed API](#continuous-data-change-feed-query).

To subscribe to a list of samples, users can:

- use the Spectra Intelligence [Alert Subscription API](#how-to-use-the-alert-subscription-api-and-data-change-feed)
- use the Spectra Intelligence [File Upload API](/SpectraIntelligence/API/Automation/tca-0202-0203) and set the query parameter `?subscribe=data_change` in their [metadata upload requests](/SpectraIntelligence/API/Automation/tca-0202-0203/#metadata-upload-request)

To unsubscribe from the Data Change Feed for previously subscribed samples or URLs, users can use the Alert Subscription API - [Unsubscribe Query](#unsubscribe-query).

## General Info about Requests/Responses

- All requests support the **format** parameter which allows changing the response format
- Default response format is **xml**, except for bulk queries, where default format is the same as the POST format
- The number of hashes in a bulk request must not be larger than one hundred (100)

## How to use the Alert Subscription API and Data Change Feed

The subscription procedure consists of two general steps.

### Step 1 - Subscribe to samples

Users submit a list of sample or URL hashes they want to track in a bulk HTTP POST request to the Alert Subscription API. The request body can be XML or JSON.

Example: submitting a list of hashes as JSON

``` bash
curl --location 'https://data.reversinglabs.com/api/subscription/data_change/v1/bulk_query/subscribe/json?format=json' \
--header 'Authorization: Basic bXNjb3R0OnRoYXQncyB3aGF0IHNoZSBzYWlkCg==' \
--header 'Content-Type: application/json' \
--data '{
  "rl": {
    "query": {
      "hash_type": "sha1",
      "hashes": [
        "a7afddb68260a60f86c02a021efba7f216c2e7cf",
        "ca03064987d3c4465f91552ba8b6a883eecfd3e5",
        "b363713a938afcd3c74603827fab79e935b2b09b"
      ]
    }
  }
}'
```

### Step 2 - Monitor changes to subscribed samples

Users can poll the Data Change Feed using HTTP GET requests to check whether there have been any changes to the samples or URLs they have subscribed to.

The Data Change Feed returns the *last_timestamp* value that can be used in the next request up until 60 seconds from the current time. The value determines the time period for which the user wants to retrieve information about changes to subscribed samples.

For example:

1. User sends a subscription request at timestamp T0.
2. The user then issues a GET request at timestamp T1.
3. The request contains the T0 timestamp as the *last_timestamp* value, and the API returns the changes that happened between T0 and T1 (if there have been any changes).

The response returned by the API contains SHA1 hashes of subscribed samples or URLs and sections that have changed in the requested period of time.

For every sample with detected changes, the following sections are included. For URLs, only `dynamic_analysis` applies.

- `xref`:                        AV scanning results have been updated.
- `sample_available`:            The sample has become available for processing.
- `malware_presence`:            The sample's Malware Presence classification, or any part of its metadata, has changed 
                                 (including changes to the sample's Malicious/Unknown/Suspicious/Known status).
- `sample_became_shareable`:     The sample became shareable and is available for download.
- `dynamic_analysis`:            Dynamic analysis report has been completed and is available for querying.

## Subscribe query

This query is used for subscribing to a list of samples or URLs for which the changed sections (if there are any) will be delivered in the Data Change Feed. To subscribe to a list of samples or URLs, the user should submit the sample or URL hashes in a POST request. All hashes in a request should be of the same type. The maximum amount of hashes that can be submitted in one request is 100.

<Button title="View OpenAPI Specification" href="/SpectraIntelligence/API-reference/#tag/automation/post/apisubscriptiondata_changev1bulk_querysubscribepost_format" />

### Request

```
POST /api/subscription/data_change/v1/bulk_query/subscribe/{post_format}
```

#### Path parameters

- `post_format`
  - Accepts the following values: `xml` or `json`. It defines the POST body format and the response format.

#### Query parameters

- `format`
  - Accepts the following values: `xml` or `json`. Can be used to override the default response format.

#### Request body

``` json5
{
  "rl": {
    "query": {
      "hash_type": "string", // sha1, sha256, or md5
      "hashes": [
        "string"
      ]
    }
  }
}
```

- `hash_type` can be *sha1*, *md5* or *sha256* for samples, and *sha1* for URLs that were seen by the Spectra Intelligence system. **Note:** Samples or URLs that have not yet been seen can be subscribed only with SHA1 hash values.
- hash values must be valid hashes defined by the `hash_type` parameter

### Response

```json5
{
  "rl": {
    "subscription_data_change": {
      "hash_type": "string", // requested hash type
      "subscribed": [ 
        "string" // list of successful subscriptions
      ],
      "invalid_hashes": [
        "string"
      ],
      "unknown_hashes": [ 
        "string" // only if MD5 or SHA256 were used
      ]
    }
  }
}
```

## Unsubscribe query

This query is used for unsubscribing from a list of samples or URLs that the user was previously subscribed to. Submitting a sample or URL hash in a POST request to this endpoint removes the associated sample or URL from the list of user's subscriptions.

The maximum amount of hashes that can be submitted in one request is 100. Changes for unsubscribed samples or URLs will no longer be delivered in the Data Change Feed.

<Button title="View OpenAPI Specification" href="/SpectraIntelligence/API-reference/#tag/automation/post/apisubscriptiondata_changev1bulk_queryunsubscribepost_format" />

### Request

```
POST /api/subscription/data_change/v1/bulk_query/unsubscribe/{post_format}
```

The request format (body, parameters) is identical to the format in the [subscribe query](#request).

### Response

```json5
{
  "rl": {
    "subscription_data_change": {
      "hash_type": "string", // requested hash type
      "unsubscribed": [ 
        "string" // list of successful unsubscriptions
      ],
      "invalid_hashes": [
        "string"
      ],
      "unknown_hashes": [ 
        "string" // only if MD5 or SHA256 were used
      ]
    }
  }
}
```

## Examples

### Subscribe to a list of SHA1 hashes

```
/api/subscription/data_change/v1/bulk_query/subscribe/json
```

``` json
{
  "rl": {
    "query": {
      "hash_type": "sha1",
      "hashes": [
        "84dbb381485e6ee9d69fbad3c6f020be88230e9b",
        "81648fd7871017eef87e0efa166e00e891a6ea9d",
        "0761c95bd639466aff2d65c1c092e2cae40f900c",
        "552ec95c8ffe3a0cb014eee508b87730461c8f26",
        "d234189a25892325746699fe2ab39a40b2d14d6e"
      ]
    }
  }
}
```

### Unsubscribe from a list of SHA1 hashes

```
/api/subscription/data_change/v1/bulk_query/unsubscribe/json
```

``` json
{
  "rl": {
    "query": {
      "hash_type": "sha1",
      "hashes": [
        "84dbb381485e6ee9d69fbad3c6f020be88230e9b",
        "81648fd7871017eef87e0efa166e00e891a6ea9d",
        "0761c95bd639466aff2d65c1c092e2cae40f900c",
        "552ec95c8ffe3a0cb014eee508b87730461c8f26",
        "d234189a25892325746699fe2ab39a40b2d14d6e"
      ]
    }
  }
}
```

## Data change feed: PULL

This query returns the next set of samples or URLs to which the user is subscribed. The starting point for this query is defined using the [START query](#data-change-feed-start).

If the user has not previously requested this query or called the START query, it will return records starting with the current timestamp. Every subsequent call will continue from the timestamp where the previous call ended.

Unless the limit parameter is specified, the query returns a maximum of 1000 records and any surplus records sharing the same timestamp. This ensures that all the records with the same timestamp will be included in the set of records. The limit parameter must not be greater than 1000.

This endpoint supports single threaded requests. Any concurrent request will be blocked until the previous one is fulfilled.

<Button title="View OpenAPI Specification" href="/SpectraIntelligence/API-reference/#tag/automation/get/apifeeddata_changev3pull" />

### Request

```
GET /api/feed/data_change/v3/pull
```

#### Query parameters

- `events`
  - Accepts a list of sections that should be included in the response, delimited by commas. If omitted, includes all values. Otherwise, list one or more of the sections:
    - `xref`
    - `sample_available`
    - `malware_presence`
    - `sample_became_shareable`
    - `dynamic_analysis`
  - Optional
- `format`
  - Specifies the response format. Supported values: 
    - `xml` (default)
    - `json`
    - `tsv`(tab-separated values, delimiter character `\t` (`0x09`))
  - Optional
- `limit`
  - Number of records to return in the response. The maximum value is 1000. Note that the response may include a little more than the requested number of records to ensure that all the records with the same timestamp are returned.
  - Optional

### Response

```json
{
  "rl": {
    "data_change_feed": {
      "entries": [],
      "last_timestamp": 0,
      "time_range": {}
    }
  }
}
```

-----------------------------------------------------------------------

`rl.data_change_feed.entries[]`

```json
{
  "record_on": "string",
  "sha1": "string",
  "updated_sections": [
    "string"
  ],
  "sha256": "string",
  "md5": "string"
}
```

- `sha1`
  - SHA1 hash value of the sample or URL
- `md5`
  - MD5 hash value of the sample, if available
- `sha256`
  - SHA256 hash value of the sample, if available
- `updated_sections`
  - See `rl.data_change_feed.entries[].updated_sections`

-----------------------------------------------------------------------

`rl.data_change_feed.entries[].updated_sections`

- `xref`
  - Changed AV scanning results
- `malware_presence`
  - Changed Malware Presence result
- `sample_became_shareable`
  - Sample become shareable and is available for download
- `dynamic_analysis`
  - Dynamic analysis report completed and available for querying

-----------------------------------------------------------------------

## Data change feed: START

This query sets the starting timestamp for the previously described *PULL query*.

<Button title="View OpenAPI Specification" href="/SpectraIntelligence/API-reference/#tag/automation/put/apifeeddata_changev3starttime_formattime_value" />

### Request

```
PUT /api/feed/data_change/v3/start/{time_format}/{time_value}
```

#### Path parameters

- `time_format` accepts these options:
  - `timestamp`: number of seconds since 1970-01-01 00:00:00 
  - `utc`: date and time in the format `YYYY-MM-DDThh:mm:ss`
- `time_value`
  - Accepts values defined by individual option set by `time_format`

## Continuous data change feed query

This query returns a set of records (samples or URLs) that the user is subscribed to from the requested timestamp onwards. Essentially it combines the START and PULL queries into one.

The feed returns up to 1000 records and any surplus records sharing the same timestamp. The response also contains the latest timestamp up to which the events are included in the response.

To fetch the next set, use the **last_timestamp** value from the response, increase it by 1 and submit it in a new request.

The response format is the same as with the [PULL query](#data-change-feed-pull).

### Request

```
GET /api/feed/data_change/v3/query/{time_format}/{time_value}
```

#### Path parameters

- `time_format` accepts these options:
  - `timestamp`: number of seconds since 1970-01-01 00:00:00 
  - `utc`: date and time in the format `YYYY-MM-DDThh:mm:ss`
- `time_value`
  - Accepts values defined by individual option set by `time_format`

#### Query parameters

- `events`
  - Accepts a list of sections that should be included in the response, delimited by commas. If omitted, includes all values. Otherwise, list one or more of the sections:
    - `xref`
    - `sample_available`
    - `malware_presence`
    - `sample_became_shareable`
    - `dynamic_analysis`
  - Optional
- `format`
  - Specifies the response format. Supported values: 
    - `xml` (default)
    - `json`
    - `tsv`(tab-separated values, delimiter character `\t` (`0x09`))
  - Optional
- `limit`
  - Number of records to return in the response. The maximum value is 1000. Note that the response may include a little more than the requested number of records to ensure that all the records with the same timestamp are returned.
  - Optional

## Examples

- Get samples you subscribed to, set response format to JSON: `/api/feed/data_change/v3/pull?format=json`
- Get samples that changed their classification: `/api/feed/data_change/v3/pull?events=malware_presence`
- Get dynamic analysis changes and set the response format: `/api/feed/data_change/v3/pull?events=dynamic_analysis&format=json`
- Set the start time for the pull query, use UTC: `/api/feed/data_change/v3/start/utc/2015-05-01T13:00:00`
