Version: 5.2.0

Service API

The Service API provides an interface for controlling processing-related functionalities on the Worker.

Users can check the status and availability of the Worker system, and retrieve file processing statistics to observe changes in system performance.

Service API Authorization

The Worker can be configured to require an authorization token to be present in every API request. The authorization token is a string of alphanumeric characters, between 30 and 60 characters in length.

The authorization token should be provided in the HTTP Authorization header. The value of this field must be in this format:

Token exampletoken

where exampletoken should be replaced with the user's own authorization token string.

If the Worker is configured not to require an authorization token, the Authorization header field will be ignored.

curl https://tiscale-worker01/srv/tiscale/v1/sysinfo -H 'Authorization: Token exampletoken'

Service API Request Format

The base URL for the service API is in the following format:

https://{server}/srv/tiscale/v1

Service API response status codes

200
- The request has succeeded.
400
- The request could not be understood by the server due to malformed syntax or missing parameters.
401
- The request requires authorization from the user.
403
- Authorization has been provided in the request, but it is not valid.
404
- The server has not found anything matching the Request-URI.
500
- The server encountered an unexpected condition which prevented it from fulfilling the request.
503
- The server is currently unable to handle the request due to a temporary overloading or maintenance of the server.
- In case the Spectra Intelligence upload limit quota was exceeded, the response will contain the "Paused - TiCloud quota exceeded" error message. To continue analyzing files, users can temporarily disable Spectra Intelligence, wait until the daily quota renews, or contact ReversingLabs support to increase their daily upload limit. When using the Quotas API endpoints on a Worker instance without a configured Spectra Intelligence account, the response code will be 503, and the error message "TiCloud is not configured."

Get the current system information

GET /srv/tiscale/v1/sysinfo

Description

Returns current system information, indicating the status of processing-related components on the Worker instance.

Request format

Request examples

curl -X GET https://tiscale-worker01/srv/tiscale/v1/sysinfo -H 'Authorization: Token exampletoken'

import requests

url = "https://tiscale-worker01/srv/tiscale/v1/sysinfo"

headers = {'Authorization': 'Token exampletoken'}

response = requests.request("GET", url, headers=headers)

print(response.text)

Response format

A successful request returns the 200 OK status code with the sysinfo structure in the response body.

Response example

{
  "load_average": [
    0.47,
    0.27,
    0.28
  ],
  "queue_size": 0,
  "max_queue_name": "tiscale_hagent_input",
  "max_queue_size": 0,
  "hagent_running": true,
  "scratch_space": 3,
  "root_space": 3
}

Response fields

Field Name	Description	Data Type
`load_average`	Average system load for the past 1, 5, and 15 minutes.	float
`queue_size`	Amount of files in the input queue.	integer
`max_queue_name`	Name of queue which currently holds the most files.	string
`max_queue_size`	Number of files in the currently largest queue.	integer
`hagent_running`	This field is no longer supported. For the time being, it always returns the `true` value.	boolean
`scratch_space`	Indicates the percentage of used disk space on the partition holding files to be processed and reports for processed files.	float
`root_space`	Indicates the percentage of used disk space on the root partition.	float

Control the Worker operating mode

Check current operating mode status

GET /srv/tiscale/v1/mode

Returns whether the Worker is currently paused or not. Setting the paused flag to true will cause the Worker to decline all new file submissions until the flag is unset.

The flag can be used to put a Worker into maintenance mode or to remove it from a pool of Workers.

Request format

Request example

curl -X GET https://tiscale-worker01/srv/tiscale/v1/mode -H 'Authorization: Token exampletoken'

Response format

The response indicates whether the Worker instance is paused (true) or not (false). If the response equals "paused": true the Worker is considered unavailable, and is not accepting any new file submissions.

Response example

{
  "paused": false
}

Change operating mode status

PUT /srv/tiscale/v1/mode

The operating mode of the Worker instance can be changed by sending a PUT request to the same Service API endpoint.

The value of the Content-Type HTTP header field must be set to application/json.

The data supplied in the request must be a JSON object with one or more fields.

Request format

Request example

Set Worker operating mode to paused

curl -X PUT https://tiscale-worker01/srv/tiscale/v1/mode -H 'Authorization: Token exampletoken' -H 'Content-Type: application/json' -d '{"paused": true}'

Response format

A successful request returns the 200 OK status code with an empty response body.

If the Content-Type HTTP header in the request is not set to application/json, the response contains an error message "Invalid POST data: expected JSON object". Users experiencing this error message should submit a new request with the incorrect HTTP header properly configured.

Check file processing availability

GET /srv/tiscale/v1/available

Description

Checks whether the Worker instance is available for processing new files.

Request format

Request example

curl -X GET https://tiscale-worker01/srv/tiscale/v1/available -H 'Authorization: Token exampletoken'

Response format

If the Worker is available, it returns the status code 200 OK with no message in the response body.

If the instance is not available (503 Service Unavailable), it's either in an error state, or paused. There are two types of paused states:

Operating mode set to paused. In this case, the response body contains the following message: paused. See Control the Worker operating mode <opmode>.
Paused due to an exceeded Spectra Intelligence quota. In this case, the response body contains the following message: paused - TiCloud quota exceeded.

Depending on how your quota is configured (number of uploads vs. total byte size uploaded; daily vs. monthly), either wait for the quota to renew automatically or contact support to modify your quota. Alternatively, remove the Spectra Intelligence account altogether (this will disable AV scans in the cloud - only static analysis will be available).

If the Worker is in an error state, for example due to a problem with system resources, the response body contains a not healthy message.

Get file processing statistics

GET /srv/tiscale/v1/stats

Description

Returns the file processing statistics for the Worker instance.

Request format

Request example

curl https://tiscale-worker01/srv/tiscale/v1/stats -H 'Authorization: Token exampletoken'

Response format

The format of the data returned is described by the stats structure.

stats := {
   "timeframes"      : ["1m",  "10m",  "1h",  "8h",  "1d"],
   "files_submitted" : [<int>, <int>, <int>, <int>, <int>],
   "bytes_submitted" : [<int>, <int>, <int>, <int>, <int>],
   "files_processed" : [<int>, <int>, <int>, <int>, <int>],
   "bytes_processed" : [<int>, <int>, <int>, <int>, <int>],
   "files_unpacked"  : [<int>, <int>, <int>, <int>, <int>],
   "bytes_unpacked"  : [<int>, <int>, <int>, <int>, <int>],
   "mean_queue_size" : [<float>, <float>, <float>, <float>, <float>],
}

Statistics information is accrued and displayed for multiple time windows. The timeframes field lists shorthand names for those time windows (1m - one minute, 1h - one hour, etc.).

ųpload_success shows the number of successful file uploads to the Worker.
upload_failure shows the number of failed file uploads to the Worker.
s3_unpacked_success shows how many children samples successfully unpacked to the S3 bucket.
s3_report_upload shows how many reports were successfully uploaded to the S3 bucket.
s3_upload shows the number of files that were successfully uploaded to the S3 bucket.
s3_unpacked_failure shows how many children samples failed to unpack to the S3 bucket.
sns_failure shows the number of Amazon SMS messages that failed to deliver.
max_mean_queue_name shows the names of queues with the largest mean amount of tasks in that timeframe.
general_service_failure shows the amount of service errors (5xx server errors).
callback_failure shows how many reports failed to upload to the Callback server.
files_submitted shows the number of files that were submitted for processing.
mean_queue_size shows the mean value of the input queue size for each of the time windows.
general_user_failure shows the number of errors caused by the user, for example by malformed API requests (4xx client errors).
files_processed and bytes_processed fields show the number of files/bytes that were successfully processed.
splunk_failure shows how many reports failed to upload to Splunk.
s3_upload_failure shows the number of unsuccessful uploads to the S3 bucket.
bytes_submitted shows the total number of bytes of all files that were submitted.
splunk_success shows how many reports were successfully uploaded to Splunk.
callback_success shows how many reports were successfully uploaded to the Callback server.
not_available is triggered when the Worker is in an unhealthy state (for example, disk usage is above the set threshold).
sns_success shows the number of successfully delivered Amazon SMS messages.
max_mean_queue_size shows the mean amount of tasks for queues listed in max_mean_queue_name.
s3_report_upload_failure shows how many reports failed to upload to the S3 bucket.
files_unpacked and bytes_unpacked fields show the number of files/bytes that were extracted (unpacked) from samples that were uploaded for processing.
file_processing_failure shows the number of files that the service failed to analyze.

Response example

A successful request returns the 200 OK status code with the list of processing statistics fields and their values in the response.

{
  "upload_success": [
    7,
    15,
    15,
    15,
    15
  ],
  "timeframes": [
    "1m",
    "10m",
    "1h",
    "8h",
    "1d"
  ],
  "upload_failure": [
    0,
    0,
    0,
    0,
    0
  ],
  "s3_unpacked_success": [
    0,
    0,
    0,
    0,
    0
  ],
  "s3_report_upload": [
    0,
    0,
    0,
    0,
    0
  ],
  "s3_upload": [
    0,
    0,
    0,
    0,
    0
  ],
  "s3_unpacked_failure": [
    0,
    0,
    0,
    0,
    0
  ],
  "sns_failure": [
    0,
    0,
    0,
    0,
    0
  ],
  "max_mean_queue_name": [
    "tiscale.hagent_input",
    "tiscale.hagent_input",
    "tiscale.hagent_input",
    "tiscale.hagent_input",
    "tiscale.hagent_input"
  ],
  "general_service_failure": [
    0,
    0,
    0,
    0,
    0
  ],
  "callback_failure": [
    0,
    0,
    0,
    0,
    0
  ],
  "bytes_processed": [
    142273712,
    304872240,
    304872240,
    304872240,
    304872240
  ],
  "files_submitted": [
    7,
    15,
    15,
    15,
    15
  ],
  "mean_queue_size": [
    0.625,
    0.08908045977011493,
    0.014432029795158285,
    0.001795228167709057,
    0.0008703953279424976
  ],
  "general_user_failure": [
    0,
    0,
    0,
    0,
    0
  ],
  "files_processed": [
    7,
    15,
    15,
    15,
    15
  ],
  "splunk_failure": [
    0,
    0,
    0,
    0,
    0
  ],
  "bytes_unpacked": [
    303859038,
    651126510,
    651126510,
    651126510,
    651126510
  ],
  "s3_upload_failure": [
    0,
    0,
    0,
    0,
    0
  ],
  "bytes_submitted": [
    142273712,
    304872240,
    304872240,
    304872240,
    304872240
  ],
  "splunk_success": [
    0,
    0,
    0,
    0,
    0
  ],
  "callback_success": [
    0,
    0,
    0,
    0,
    0
  ],
  "not_available": [
    0,
    0,
    0,
    0,
    0
  ],
  "sns_success": [
    0,
    0,
    0,
    0,
    0
  ],
  "max_mean_queue_size": [
    0.625,
    0.08908045977011493,
    0.014432029795158285,
    0.001795228167709057,
    0.0008703953279424976
  ],
  "s3_report_upload_failure": [
    0,
    0,
    0,
    0,
    0
  ],
  "files_unpacked": [
    56,
    120,
    120,
    120,
    120
  ],
  "file_processing_failure": [
    0,
    0,
    0,
    0,
    0
  ]
}

Service API Authorization​

Service API Request Format​

Service API response status codes​

Get the current system information​

Description​

Request format​

Response format​

Control the Worker operating mode​

Check current operating mode status​

Request format​

Response format​

Change operating mode status​

Request format​

Response format​

Check file processing availability​

Description​

Request format​

Response format​

Get file processing statistics​

Description​

Request format​

Response format​

Service API Authorization

Service API Request Format

Service API response status codes

Get the current system information

Description

Request format

Response format

Control the Worker operating mode

Check current operating mode status

Request format

Response format

Change operating mode status

Request format

Response format

Check file processing availability

Description

Request format

Response format

Get file processing statistics

Description

Request format

Response format