Vertical feeds search (TCA-0312-0316, 0318)
This service can be used to retrieve information about new malware samples from ReversingLabs Targeted and Industry-Specific File Indicator Feeds by searching for malware family names.
The feeds are specialized collections of malware families that are known to have significant impact within specific industries (Retail, Financial), as well as of malware families that share a common trait (exploits, ransomware). ReversingLabs carefully selects malware families for each feed based on public and internal research.
The list of malware families for each feed is continually updated by ReversingLabs analysts. An excerpt from the list is available in the Sample Malware Family Names section. For the complete, accurate, and up-to-date list of all malware family names in Targeted and Industry-Specific File Indicator Feeds, it is recommended to obtain access to Statistics APIs (TCA 0307-0311, 0317)
Using the API described in this document, it is possible to search for the following information:
- new malware hashes belonging to an APT tool
- new malware hashes belonging to a banking Trojan or other malware family targeting the financial sector
- new malware hashes belonging to a POS malware or other malware family targeting the retail sector
- new malware hashes belonging to a ransomware family
- new exploit hashes related to a CVE identifier
- new malware hashes that contain embedded configuration data
Apart from SHA1 hashes, results include information about other file properties: MD5 and SHA256 hashes, file size, file type, download availability of the samples, first seen, last seen (UTC), container hash and time when the file was added into the feed.
To make sense of that data, ReversingLabs is creating weekly statistics and providing them via this API (Statistics API).
Example Malware Family Names
The following malware family names can be used to test the API described in this document.
For the complete, accurate, and up-to-date list of all malware family names in Targeted and Industry-Specific File Indicator Feeds, it is recommended to obtain access to Statistics APIs (TCA 0307-0311, 0317).
| Code | Feed Name | Malware Family Names |
|---|---|---|
| TCA-0312 | APT (Advanced Persistent Threats) | CosmicDuke, CozyBear, Stuxnet, Hellsing |
| TCA-0313 | Financial Services Malware | Alice, Dorkbot, Ramnit, Ripper |
| TCA-0314 | Retail Sector Malware | AbaddonPOS, ChewBacca, Katrina, Poseidon |
| TCA-0315 | Ransomware | BitCrypt, Nanolocker, NotPetya, WannaCry |
| TCA-0316 | CVE Exploits | CVE-2008-4844, CVE-2014-0495, CVE-2017-0147, CVE-2017-8291 |
| TCA-0318 | Malware Configuration | DarkComet, PoisonIvy, XtremeRAT, CyberGate |
Family Name Index Query
The query allows searching for new malware hashes from ReversingLabs Targeted and Industry-Specific File Indicator Feeds by malware family name.
Note that the malware family name must match at least one family name from the feeds in order to get the desired results. In other words, if the queried family name is not found in any of the feeds, no detections will be fetched.
Since the list of malware family names in each feed changes daily, it is recommended to obtain access to Vertical Feeds Statistics APIs (TCA 0307-0311, 0317) to get the complete and accurate information.
For the purposes of testing this query, refer to the list of Sample Malware Family Names.
Fetch detections since specified time
This query returns a recordset of entries from one of Targeted and Industry-Specific File Indicator Feeds in the specified time range for the given malware family name, provided that the family name is found in one of the feeds.
Entries are included in the response based on the time when they were added to a particular feed. The time value provided in the query defines the starting point of the time range. If an entry has been added to one of the feeds in that time range, it is included in the response.
Request
GET /api/feed/malware/detection/family/v2/index/family_name/search/{family_name}/from/{time_format}/{time_value}?[format=xml|json]&[count=(1-1000)]
Path parameters:
family_name- Case-sensitive parameter; accepts a malware family name or a CVE identifier
- Required
time_format- Specifies the time format for the
time_valueparameter. Supported values: timestamp - Unix epoch time as the number of seconds since 1970-01-01 00:00:00; utc - YYYY-MM-DDThh:mm:ss - Required
- Specifies the time format for the
time_value- Accepts values formatted according to the format set in the
time_formatparameter. The latest allowed time value is 60 seconds before the current time. - Required
- Accepts values formatted according to the format set in the
Query parameters:
format- Specifies the response format. Supported values: xml (default), json
- Optional
count- The number of hashes to return in the response. It has to be an integer in the range from 1 to 1000 ( 1 <= count <= 1000 ). The default value of the parameter is 100.
- Optional
The time range is defined as the period between the time specified by the time_value parameter and the time when the request is made.
The earliest time value that the user can specify is 365 days before the current time. The latest time value is 60 seconds before the current time. Specifying a time value outside of these limits will return the status code 400 Bad Request.
The response can contain up to 1000 results. If the number of samples with the last timestamp included in the response goes above this limit, the results will be cut off.
To get the next page of results, provide the to value from the previous request as the timestamp value of the next request.
This will cause some result duplication (specifically, records whose record_on value matches the to timestamp will reappear on the second page), but it will also return any remaining records with the matching timestamp that didn't fit on the first page.
Setting the count value too low may result in from and to timestamps being the same, which will cause a response loop.
All time values are in UTC, independent of the input format.
Fetch the latest detections
This query returns a recordset from Targeted and Industry-Specific File Indicator Feeds consisting of the latest entries for the given family name, provided that the family name is found in one of the feeds.
Request
GET /api/feed/malware/detection/family/v2/index/family_name/search/{family_name}/latest?[format=xml|json]&[count=(1-1000)]
Path parameters:
family_name- case-sensitive and accepts a malware family name or a CVE identifier
- Required
count- Specify how many latest hashes to return in the response. It has to be an integer in the range from 1 to 1000 ( 1 <= count <= 1000 ). The default value of the parameter is 100.
- Optional
Query parameters:
format- Specifies the response format. Supported values: xml (default), json
- Optional Submitting a GET request to the given URL with no optional request arguments will provide the latest 100 detections for the given family name.
The count parameter supports all integers between 1 and 1000, including 1 and 1000. Providing count with anything other than integers in the given range will result in a "400 Bad Request" response.
The feed will return 100 records if the count parameter is not provided. Otherwise, it will return the requested number of detections, starting from the most recent record in the data store.
The most recent entry value is 60 seconds before the current time.
Response Format
{
"rl": {
"feed": {
"name": "string",
"time_range": {
"from": 0,
"to": 0
},
"entries": [],
"family_name": "string",
"categories": [
"string"
]
}
}
}
family_name- the requested family name or CVE identifier
categories- list of categories this family_name belongs to
time_range- Date and time when the item was first seen (UTC)
time_range \> from- timestamp from the latest record_on field from entries
time_range \> to- time when the request was initiated
name- name of the feed
entries- Contains a list of entries with fields
rl.feed.entries[]
{
"record_on": 0,
"sha1": "string",
"priority": 0,
"family_name": "string",
"sample_available": true,
"container_hash": "string",
"sha256": "string",
"md5": "string",
"sample_size": 0,
"sample_type": "string",
"first_seen_on": 0,
"last_seen_on": 0,
"tags": {
"actors": [
"string"
]
}
}
sha1- sample SHA1
sample_available- boolean value indicating whether the sample is physically available for download, but not if the file is shareable (available for public download). Regardless of this value, use the Sample Download API to attempt a file download. If the Sample Download API returns a 403 error, that indicates that this sample is not available for download, or is private.
sample_size- integer value defining the sample size
priority- integer value (1-5) describing the importance of the entry for a particular industry (category). The value of 1 indicates the lowest priority, and 5 the highest. Lower values mean that the particular malware family is connected to some actor or other malware families related to the industry; not directly affecting the industry, but relevant to it
sample_type- string value describing the sample type (e.g. "Binary/Archive")
record_on- time when the record was added to the feed
family_name- malware family name
container_hash- container SHA1; if this information is the sample itself, then this is identical to SHA1
first_seen_on- time when this hash was first seen in the system (UTC)
last_seen_on- time when this hash was last seen in the system (UTC)
sha256- sample SHA256, if it exists
md5- sample md5, if it exists
tags- a dictionary that contains tag_types as keys and tag_list as the value for each tag_type that is found in the configuration for a given detection
Examples
Format query field
Here the format for query response varies:
Fetching the first 100 entries starting from 2017-03-20 00:00:00 GMT till 2016-10-30 23:59:59 GMT for the ChewBacca family name:
/api/feed/malware/detection/family/v2/index/family_name/search/ChewBacca/from/timestamp/1489968000
/api/feed/malware/detection/family/v2/index/family_name/search/ChewBacca/from/utc/2017-03-20T00:00:00
Fetching the first 1000 entries starting from 2017-03-20 00:00:00 GMT till 2016-10-30 23:59:59 GMT for the Stuxnet family name
/api/feed/malware/detection/family/v2/index/family_name/search/Stuxnet/from/timestamp/1489968000?count=1000
/api/feed/malware/detection/family/v2/index/family_name/search/Stuxnet/from/utc/2017-03-20T00:00:00?count=1000
Fetching the 100 latest entries or entries dating up to the last 30 days for the Ramnit family name
/api/feed/malware/detection/family/v2/index/family_name/search/Ramnit/latest
Fetching the 10 latest entries or entries dating up to the last 30 days for the CozyBear family name
/api/feed/malware/detection/family/v2/index/family_name/search/CozyBear/latest?count=10
Fetching the 10 latest entries or entries dating up to the last 30 days for the NotPetya family name, with requested response in the JSON format
/api/feed/malware/detection/family/v2/index/family_name/search/NotPetya/latest?count=10&format=json
Retrieving the next page in feed
The following two examples retrieve the first and second page of results for the ChewBacca family using the timestamp parameter.
The first request will search for entries newer than the given timestamp (1490465964) and return 2 entries per page (in practical use, this number should be higher, 2 was chosen as an example):
GET /api/feed/malware/detection/family/v2/index/family_name/search/ChewBacca/from/timestamp/1490465964?count=2&format=json
Response
{
"rl": {
"feed": {
"family_name": "ChewBacca",
"entries": [
{
"family_name": "ChewBacca",
"sha1": "7a6b3(...)e0c35",
"record_on": 1490466315,
"first_seen_on": 1490465598,
"last_seen_on": 1490466240,
"sample_type": "PE/Exe",
"priority": 5,
"sample_size": 1326080,
"sample_available": true,
"container_hash": "7a6b3(...)e0c35",
"sha256": "0d596(...)9b491",
"md5": "7ed1cfcf70f2262b14c79ed19ece8bde"
},
{
"family_name": "ChewBacca",
"sha1": "a04cf(...)8c2b5",
"record_on": 1490467370,
"first_seen_on": 1490466445,
"last_seen_on": 1490467320,
"sample_type": "PE/Exe",
"priority": 5,
"sample_size": 1326080,
"sample_available": true,
"container_hash": "a04cf(...)8c2b5",
"sha256": "a37c9(...)f60b8",
"md5": "88458d3d784b54097786ee9f2756f405"
}
],
"name": "Malware Detection Family Feed",
"categories": [
"retail"
],
"time_range": {
"to": 1490467370,
"from": 1490465964
}
}
}
}
After receiving the response, the timestamp value (to timestamp) from the time_range.to field can be used for requesting the next page of results.
To get the next page of results, provide the to value from the previous request as the timestamp value of the next request.
This will cause some result duplication (specifically, records whose record_on value matches the to timestamp will reappear on the second page), but it will also return any remaining records with the matching timestamp that didn't fit on the first page.
Setting the count value too low may result in from and to timestamps being the same, which will cause a response loop.
GET /api/feed/malware/detection/family/v2/index/family_name/search/ChewBacca/from/timestamp/1490466369?count=2&format=json
This will return page 2 with two entries:
{
"rl": {
"feed": {
"family_name": "ChewBacca",
"entries": [
{
"family_name": "ChewBacca",
"sha1": "a04cf(...)8c2b5",
"record_on": 1490467370,
"first_seen_on": 1490466445,
"last_seen_on": 1490467320,
"sample_type": "PE/Exe",
"priority": 5,
"sample_size": 1326080,
"sample_available": true,
"container_hash": "a04cf(...)8c2b5",
"sha256": "a37c9(...)f60b8",
"md5": "88458d3d784b54097786ee9f2756f405"
},
{
"family_name": "ChewBacca",
"sha1": "96aef904580e435fbdf3b1e22d8cd8dde351b4e6",
"record_on": 1490468400,
"first_seen_on": 1490376320,
"last_seen_on": 1490468340,
"sample_type": "PE/Exe",
"priority": 5,
"sample_size": 1837056,
"sample_available": true,
"container_hash": "96aef(...)1b4e6",
"sha256": "0d9e3e(...)649bd",
"md5": "b1898d0e1998ce17aa7e20e328ac5d14"
}
],
"name": "Malware Detection Family Feed",
"categories": [
"retail"
],
"time_range": {
"to": 1490468400,
"from": 1490466369
}
}
}
}