# Network Indicators Feed (TCF-0301) — Spectra Intelligence

> Consume TCF-0301 for a continuous stream of malicious URLs identified by Spectra Intelligence, including associated file SHA1/SHA256 hashes and threat names.

# Network IOCs feed (TCF-0301)

This service provides a continuous feed of malicious URLs identified by Spectra Intelligence.

When a URL is observed delivering malicious files, the feed also includes relevant metadata for the downloaded file, such as the SHA1 hash, SHA256, and the ReversingLabs threat name.

The feed stores records for the last 365 days.

The feed returns a maximum of 1000 records per request, and provides the latest queried timestamp up to which the records are included. In your next request, increment that timestamp by 1 to fetch the next batch of records.

If the requested timestamp is not within the last 365 days, the service will respond with the status code **400 Bad Request**.

## Continuous Feed Query

<Button title="View OpenAPI Specification" href="/SpectraIntelligence/API-reference/#tag/network-indicators/get/api/feed/malware_uri/v1/query/{time_format}/{time_value}" />

### Request

```
GET /api/feed/malware_uri/v1/query/{time_format}/{time_value}[?format=xml|json|tsv]
```

- ``time_format``
  - Format in which the time value will be specified. Supported values are: **timestamp** (number of seconds since 1970-01-01 00:00:00), **utc** (UTC date in the *YYYY-MM-DDThh:mm:ss* format). This will also determine which format will be used in the `last_timestamp` field of the response.
  - Required
- ``time_value``
  - Accepts values in the format set by `time_format`
  - Required
- ``format``
  - Specifies the format in which the resulting data will be returned. Supported values are: **xml** (default), **json**, **tsv** (Tab Separated Values, delimiter character `\t 0x09`)
  - Optional

To retrieve at most 1000 latest records, use the following query:

```
GET /api/feed/malware_uri/v1/query/latest[?format=xml|json|tsv]
```

### Response

Records returned from third party sources contain only the `uri`, `threat_level` and `uri_type` fields.

``` json
{
  "rl": {
    "malware_uri_feed": {
      "entries": [
        {
          "record_on": "string",
          "uri": "string",
          "samples": [
            {
              "threat_name": "string",
              "sha1": "string",
              "sha256": "string"
            }
          ],
          "uri_type": "string"
        }
      ],
      "last_timestamp": "string",
      "time_range": {
        "to": "string",
        "from": "string"
      }
    }
  }
}
```

-----------------------------------------------------------------------

`rl.malware_uri_feed.entries[]`

- `record_on`
  - When the record was included in the feed. Format: utc (YYYY-MM-DDThh:mm:ss).
- `samples`
  - List of samples with optional threat names. Contains the following:
    - `sha1`: SHA1 hash of the sample, for example 9300addf4d28d3c1419d0080fbb415bb50e8a19c
    - `sha256`: SHA256 hash of the sample, for example 9f0da89c727b6efb3d2a3f845254bf89c626980a70a5817918d9c43ae65858c0
    - `threat_name`: optional threat name, for example Win32.Trojan.Dinwod
- `uri`
A URL that we downloaded a malicious file from, or a third party reputation provider found it malicious.
 - `threat_level`
   - Malware severity indicator expressed as an integer between 0 and 5. Values from `1` to `5` indicate threats from lowest to highest severity, `0` is reserved for known URIs. For unknown URIs, this value is omitted.
   - In real-world situations, threat level values are typically interpreted in the following way:
     - Threat Level 4, 5 - immediate response required (e.g., different types of Trojans, URI found on a blocklist or matches a known malware regex)
     - Threat Level 2, 3 - should be examined within 24 hours (e.g., first stage exploits, URLs with homoglyph variations)
     - Threat Level 1 - not urgent, but should be periodically reviewed (e.g. Adware / PUA, misleading subdomains).
- `uri_type`
  - This field is included for the purpose of backward compatibility. Always equal to url.

### Examples

**Fetching all new URIs with changes from 2023-01-01 00:00:00**

```
/api/feed/malware_uri/v1/query/utc/2023-01-01T00:00:00
```

**Fetching all new URIs with changes from 2023-01-01 00:00:00 using the timestamp format; response in JSON, XML, and TSV format**

```
/api/feed/malware_uri/v1/query/timestamp/1672527600?format=json
/api/feed/malware_uri/v1/query/timestamp/1672527600?format=xml
/api/feed/malware_uri/v1/query/timestamp/1672527600?format=tsv
```

**Fetching the first available changes**

```
/api/feed/malware_uri/v1/query/timestamp/0
```

**Fetching all latest changes in XML, JSON, and TSV format**

```
/api/feed/malware_uri/v1/query/latest?format=json
/api/feed/malware_uri/v1/query/latest?format=xml
/api/feed/malware_uri/v1/query/latest?format=tsv
```

## PULL Query

For a given point in time, this query returns a list of malicious URIs that have been detected in the Spectra Intelligence system.

The starting point for this query is defined using the `start` query. If the user has not previously requested this query, nor has the START query been called, it will return records starting with the current timestamp. Every subsequent call will continue from the timestamp where the previous call ended. In case that the timestamp of the previous call is older than 365 days, the subsequent call will autocorrect this timestamp to the oldest available (i.e. `current - 365 days`), and corresponding records will be returned.

When using the pull query continuously, it is useful to implement a pause between API requests to allow more results to accumulate in the feed, increasing the number of results returned per API request and reducing the total number of API calls needed. However, when catching up from a distant past timestamp, minimizing the pause between requests will speed up the process. Once the timestamp from the latest API response matches the current time, both methods (with and without pause) yield a similar number of records per unit of time.

If you want to keep fetching the latest available records as they appear in the feed, you can use a short pause (e.g., 1 second) between requests. And if you need to catch up on a large backlog of data from the past, it is better to make continuous API calls without pausing.

Unless the **limit** parameter is specified, the feed returns up to 1000 records and any surplus records sharing the same timestamp. That ensures all the records with the same timestamp will be included in the recordset. The **limit** parameter must not be greater than 1000.

This endpoint is built to be queried by a single thread (single instance). Any concurrent requests will be blocked until the previous request is fulfilled.

<Button title="View OpenAPI Specification" href="/SpectraIntelligence/API-reference/#tag/network-indicators/get/api/feed/cf/malware_uri/v1/query/pull" />

### Request

```
GET /api/feed/cf/malware_uri/v1/query/pull[?format=xml|json][&limit=N]
```

- ``format``
  - Parameter defining the format in which the resulting data will be returned. Supported values are: **xml** (default), **json**.
  - Optional
- ``limit``
  - The number of records to return in the response. The maximum and default value is 1000. Note that the response may include more records than requested to ensure that all records with the same timestamp are returned.
  - Optional

### Response

The response format is the same as in the `feed-query`.

### Examples

**Fetching all new URI changes since last pull in XML and JSON format**

```
/api/feed/cf/malware_uri/v1/query/pull?format=json
/api/feed/cf/malware_uri/v1/query/pull?format=xml
```

**Fetching a smaller batch of results per request**

```
/api/feed/cf/malware_uri/v1/query/pull?limit=100
/api/feed/cf/malware_uri/v1/query/pull?format=json&limit=100
```

## START Query

This query sets the starting timestamp for the previously described `pull-query`. It is used to reprocess an already pulled time span, obtain data from an older period that was not previously retrieved, or skip ahead to a more recent timestamp when older data is no longer relevant. Resetting the start time in these cases helps manage the data retrieval process effectively.

The starting timestamp must be within the last 365 days, otherwise the service will respond with the status code **400 Bad Request**.

<Button title="View OpenAPI Specification" href="/SpectraIntelligence/API-reference/#tag/network-indicators/put/api/feed/cf/malware_uri/v1/query/start/{time_format}/{time_value}" />

### Request

```
PUT /api/feed/cf/malware_uri/v1/query/start/[time_format]/[time_value]
```

- ``time_format``
  - Format in which the time value will be specified. Supported values are: **timestamp** - number of seconds since 1970-01-01 00:00:00; **utc** - UTC date in the *YYYY-MM-DDThh:mm:ss* format
  - Required
- ``time_value``
  - Accepts values in the format set by time_format
  - Required

### Examples

**Set the start time using UTC format**

```
/api/feed/cf/malware_uri/v1/query/start/utc/2023-01-01T00:00:00
```

**Set the start time using the timestamp format**

```
/api/feed/cf/malware_uri/v1/query/start/timestamp/1672527600
```

### Response

A successful query returns an **HTTP 200 OK** message with an empty response body.
