Service API

The Service API provides an interface for controlling processing-related functionalities on the Worker.

Users can check the status and availability of the Worker system, and retrieve file processing statistics to observe changes in system performance.

Service API Authorization

The Worker can be configured to require an authorization token to be present in every API request. The authorization token is a string of alphanumeric characters, between 30 and 60 characters in length.

The authorization token should be provided in the HTTP Authorization header. The value of this field must be in this format:

Token exampletoken

where exampletoken should be replaced with the user’s own authorization token string.

If the Worker is configured not to require an authorization token, the Authorization header field will be ignored.

Example curl request
curl https://tiscale-worker01/srv/tiscale/v1/sysinfo -H 'Authorization: Token exampletoken'

Service API Request Format

The base URL for the service API is in the following format:

https://{server}/srv/tiscale/v1

Replace {server} with the host name or the IP address of the Worker.

Service API Response Status Codes

Code

Name

Description

200

OK

The request has succeeded.

400

Bad Request

The request could not be understood by the server due to malformed syntax or missing parameters.

401

Unauthorized

The request requires authorization from the user.

403

Forbidden

Authorization has been provided in the request, but it is not valid.

404

Not Found

The server has not found anything matching the Request-URI.

500

Internal Server Error

The server encountered an unexpected condition which prevented it from fulfilling the request.

503

Service Unavailable

The server is currently unable to handle the request due to a temporary overloading or maintenance of the server.

In case the Spectra Intelligence upload limit quota was exceeded, the response will contain the “Paused - TiCloud quota exceeded” error message. To continue analyzing files, users can temporarily disable Spectra Intelligence, wait until the daily quota renews, or contact ReversingLabs support to increase their daily upload limit. When using the Quotas API endpoints on a Worker instance without a configured Spectra Intelligence account, the response code will be 503, and the error message “TiCloud is not configured.”

Get the current system information

GET

http(s)://{server}/srv/tiscale/v1/sysinfo

Description

Returns current system information, indicating the status of processing-related components on the Worker instance.

Request format

Request examples

cURL

curl -X GET \
   https://tiscale-worker01/srv/tiscale/v1/sysinfo \
   -H 'Authorization: Token exampletoken'

Python

import requests

url = "https://tiscale-worker01/srv/tiscale/v1/sysinfo"

headers = {'Authorization': 'Token exampletoken'}

response = requests.request("GET", url, headers=headers)

print(response.text)

Response format

A successful request returns the 200 OK status code with the sysinfo structure in the response body.

Response example

{
   "load_average": [
      0.47,
      0.27,
      0.28
   ],
   "queue_size": 0,
   "max_queue_name": "tiscale_hagent_input",
   "max_queue_size": 0,
   "hagent_running": true,
   "scratch_space": 3,
   "root_space": 3
}

Response fields

Field Name

Description

Data Type

load_average

Average system load for the past 1, 5, and 15 minutes.

float

queue_size

Amount of files in the input queue.

integer

max_queue_name

Name of queue which currently holds the most files.

string

max_queue_size

Number of files in the currently largest queue.

integer

hagent_running

This field is no longer supported. For the time being, it always returns the true value.

boolean

scratch_space

Indicates the percentage of used disk space on the partition holding files to be processed and reports for processed files.

float

root_space

Indicates the percentage of used disk space on the root partition.

float

Control the Worker operating mode

Check current operating mode status

GET

http(s)://{server}/srv/tiscale/v1/mode

Returns whether the Worker is currently paused or not. Setting the paused flag to true will cause the Worker to decline all new file submissions until the flag is unset.

The flag can be used to put a Worker into maintenance mode or to remove it from a pool of Workers.

Request format

Request example

cURL

curl -X GET \
   https://tiscale-worker01/srv/tiscale/v1/mode \
   -H 'Authorization: Token exampletoken'

Response format

The response indicates whether the Worker instance is paused (true) or not (false). If the response equals "paused": true the Worker is considered unavailable, and is not accepting any new file submissions.

Response example

{
   "paused": false
}

…

Change operating mode status

PUT

http(s)://{server}/srv/tiscale/v1/mode

The operating mode of the Worker instance can be changed by sending a PUT request to the same Service API endpoint.

The value of the Content-Type HTTP header field must be set to application/json.

The data supplied in the request must be a JSON object with one or more fields.

Request format

Request example

cURL - Set Worker operating mode to paused

curl -X PUT \
https://tiscale-worker01/srv/tiscale/v1/mode \
-H 'Authorization: Token exampletoken' \
-H 'Content-Type: application/json' \
-d '{
      "paused": true
   }'

Response format

A successful request returns the 200 OK status code with an empty response body.

If the Content-Type HTTP header in the request is not set to application/json, the response contains an error message “Invalid POST data: expected JSON object”. Users experiencing this error message should submit a new request with the incorrect HTTP header properly configured.

Check file processing availability

GET

http(s)://{server}/srv/tiscale/v1/available

Description

Checks whether the Worker instance is available for processing new files.

Request format

Request example

cURL

curl -X GET \
   https://tiscale-worker01/srv/tiscale/v1/available \
   -H 'Authorization: Token exampletoken'

Response format

If the Worker is available, it returns the status code 200 OK with no message in the response body.

If the instance is not available (503 Service Unavailable), it’s either in an error state, or paused. There are two types of paused states:

  1. Operating mode set to paused. In this case, the response body contains the following message: paused. See Control the Worker operating mode.

  2. Paused due to an exceeded Spectra Intelligence quota. In this case, the response body contains the following message: paused - TiCloud quota exceeded.

    Depending on how your quota is configured (number of uploads vs. total byte size uploaded; daily vs. monthly), either wait for the quota to renew automatically or contact support to modify your quota. Alternatively, remove the Spectra Intelligence account altogether (this will disable AV scans in the cloud - only static analysis will be available).

If the Worker is in an error state, for example due to a problem with system resources, the response body contains a not healthy message.

Get file processing statistics

GET

http(s)://{server}/srv/tiscale/v1/stats

Description

Returns the file processing statistics for the Worker instance.

Request format

Request example

cURL

curl -X GET \
   https://tiscale-worker01/srv/tiscale/v1/stats \
   -H 'Authorization: Token exampletoken'

Response format

The format of the data returned is described by the stats structure.

stats := {
   "timeframes"      : ["1m",  "10m",  "1h",  "8h",  "1d"],
   "files_submitted" : [<int>, <int>, <int>, <int>, <int>],
   "bytes_submitted" : [<int>, <int>, <int>, <int>, <int>],
   "files_processed" : [<int>, <int>, <int>, <int>, <int>],
   "bytes_processed" : [<int>, <int>, <int>, <int>, <int>],
   "files_unpacked"  : [<int>, <int>, <int>, <int>, <int>],
   "bytes_unpacked"  : [<int>, <int>, <int>, <int>, <int>],
   "mean_queue_size" : [<float>, <float>, <float>, <float>, <float>],
}

Statistics information is accrued and displayed for multiple time windows. The timeframes field lists shorthand names for those time windows (1m - one minute, 1h - one hour, etc.).

  • ųpload_success shows the number of successful file uploads to the Worker.

  • upload_failure shows the number of failed file uploads to the Worker.

  • s3_unpacked_success shows how many children samples successfully unpacked to the S3 bucket.

  • s3_report_upload shows how many reports were successfully uploaded to the S3 bucket.

  • s3_upload shows the number of files that were successfully uploaded to the S3 bucket.

  • s3_unpacked_failure shows how many children samples failed to unpack to the S3 bucket.

  • sns_failure shows the number of Amazon SMS messages that failed to deliver.

  • max_mean_queue_name shows the names of queues with the largest mean amount of tasks in that timeframe.

  • general_service_failure shows the amount of service errors (5xx server errors).

  • callback_failure shows how many reports failed to upload to the Callback server.

  • files_submitted shows the number of files that were submitted for processing.

  • mean_queue_size shows the mean value of the input queue size for each of the time windows.

  • general_user_failure shows the number of errors caused by the user, for example by malformed API requests (4xx client errors).

  • files_processed and bytes_processed fields show the number of files/bytes that were successfully processed.

  • splunk_failure shows how many reports failed to upload to Splunk.

  • s3_upload_failure shows the number of unsuccessful uploads to the S3 bucket.

  • bytes_submitted shows the total number of bytes of all files that were submitted.

  • splunk_success shows how many reports were successfully uploaded to Splunk.

  • callback_success shows how many reports were successfully uploaded to the Callback server.

  • not_available is triggered when the Worker is in an unhealthy state (for example, disk usage is above the set threshold).

  • sns_success shows the number of successfully delivered Amazon SMS messages.

  • max_mean_queue_size shows the mean amount of tasks for queues listed in max_mean_queue_name.

  • s3_report_upload_failure shows how many reports failed to upload to the S3 bucket.

  • files_unpacked and bytes_unpacked fields show the number of files/bytes that were extracted (unpacked) from samples that were uploaded for processing.

  • file_processing_failure shows the number of files that the service failed to analyze.

Response example

A successful request returns the 200 OK status code with the list of processing statistics fields and their values in the response.

{
   "upload_success": [
      7,
      15,
      15,
      15,
      15
   ],
   "timeframes": [
      "1m",
      "10m",
      "1h",
      "8h",
      "1d"
   ],
   "upload_failure": [
      0,
      0,
      0,
      0,
      0
   ],
   "s3_unpacked_success": [
      0,
      0,
      0,
      0,
      0
   ],
   "s3_report_upload": [
      0,
      0,
      0,
      0,
      0
   ],
   "s3_upload": [
      0,
      0,
      0,
      0,
      0
   ],
   "s3_unpacked_failure": [
      0,
      0,
      0,
      0,
      0
   ],
   "sns_failure": [
      0,
      0,
      0,
      0,
      0
   ],
   "max_mean_queue_name": [
      "tiscale.hagent_input",
      "tiscale.hagent_input",
      "tiscale.hagent_input",
      "tiscale.hagent_input",
      "tiscale.hagent_input"
   ],
   "general_service_failure": [
      0,
      0,
      0,
      0,
      0
   ],
   "callback_failure": [
      0,
      0,
      0,
      0,
      0
   ],
   "bytes_processed": [
      142273712,
      304872240,
      304872240,
      304872240,
      304872240
   ],
   "files_submitted": [
      7,
      15,
      15,
      15,
      15
   ],
   "mean_queue_size": [
      0.625,
      0.08908045977011493,
      0.014432029795158285,
      0.001795228167709057,
      0.0008703953279424976
   ],
   "general_user_failure": [
      0,
      0,
      0,
      0,
      0
   ],
   "files_processed": [
      7,
      15,
      15,
      15,
      15
   ],
   "splunk_failure": [
      0,
      0,
      0,
      0,
      0
   ],
   "bytes_unpacked": [
      303859038,
      651126510,
      651126510,
      651126510,
      651126510
   ],
   "s3_upload_failure": [
      0,
      0,
      0,
      0,
      0
   ],
   "bytes_submitted": [
      142273712,
      304872240,
      304872240,
      304872240,
      304872240
   ],
   "splunk_success": [
      0,
      0,
      0,
      0,
      0
   ],
   "callback_success": [
      0,
      0,
      0,
      0,
      0
   ],
   "not_available": [
      0,
      0,
      0,
      0,
      0
   ],
   "sns_success": [
      0,
      0,
      0,
      0,
      0
   ],
   "max_mean_queue_size": [
      0.625,
      0.08908045977011493,
      0.014432029795158285,
      0.001795228167709057,
      0.0008703953279424976
   ],
   "s3_report_upload_failure": [
      0,
      0,
      0,
      0,
      0
   ],
   "files_unpacked": [
      56,
      120,
      120,
      120,
      120
   ],
   "file_processing_failure": [
      0,
      0,
      0,
      0,
      0
   ]
}