Correlation engine APIs

Correlation engine APIs allow you to create new correlation rules, view findings and correlations within a certain time window, and perform other tasks.

Create correlation rules between log types

You can use the following API to create correlation rules:

  1. POST /_plugins/_security_analytics/correlation/rules

copy

Request fields

FieldTypeDescription
indexStringThe name of the index used as the log source.
queryStringThe query used to filter security logs for correlation.
categoryStringThe log type associated with the log source.

Example request

  1. POST /_plugins/_security_analytics/correlation/rules
  2. {
  3.   "correlate": [
  4.     {
  5.       "index": "vpc_flow",
  6.       "query": "dstaddr:4.5.6.7 or dstaddr:4.5.6.6",
  7.       "category": "network"
  8.     },
  9.     {
  10.       "index": "windows",
  11.       "query": "winlog.event_data.SubjectDomainName:NTAUTHORI*",
  12.       "category": "windows"
  13.     },
  14.     {
  15.       "index": "ad_logs",
  16.       "query": "ResultType:50126",
  17.       "category": "ad_ldap"
  18.     },
  19.     {
  20.       "index": "app_logs",
  21.       "query": "endpoint:/customer_records.txt",
  22.       "category": "others_application"
  23.     }
  24.   ]
  25. }

copy

Example response

  1. {
  2.   "_id": "DxKEUIkBpIjg64IK4nXg",
  3.   "_version": 1,
  4.   "rule": {
  5.     "name": null,
  6.     "correlate": [
  7.       {
  8.         "index": "vpc_flow",
  9.         "query": "dstaddr:4.5.6.7 or dstaddr:4.5.6.6",
  10.         "category": "network"
  11.       },
  12.       {
  13.         "index": "windows",
  14.         "query": "winlog.event_data.SubjectDomainName:NTAUTHORI*",
  15.         "category": "windows"
  16.       },
  17.       {
  18.         "index": "ad_logs",
  19.         "query": "ResultType:50126",
  20.         "category": "ad_ldap"
  21.       },
  22.       {
  23.         "index": "app_logs",
  24.         "query": "endpoint:/customer_records.txt",
  25.         "category": "others_application"
  26.       }
  27.     ]
  28.   }
  29. }

copy

Response fields

FieldTypeDescription
_idStringThe Id for the new rule.

List all findings and correlations within a certain time window

You can use the following API to list all findings and their correlations within a certain time window:

  1. GET /_plugins/_security_analytics/correlations?start_timestamp=<start time in milliseconds>&end_timestamp=<end time in milliseconds>

copy

Query parameters

ParameterTypeDescription
start_timestampNumberStart time for the time window, in milliseconds.
end_timestampNumberEnd time for the time window, in milliseconds.

Example request

  1. GET /_plugins/_security_analytics/correlations?start_timestamp=1689289210000&end_timestamp=1689300010000

copy

Example response

  1. {
  2.   "findings": [
  3.     {
  4.       "finding1": "931de5f0-a276-45d5-9cdb-83e1045a3630",
  5.       "logType1": "network",
  6.       "finding2": "1e6f6a12-83f1-4a38-9bb8-648f196859cc",
  7.       "logType2": "test_windows",
  8.       "rules": [
  9.         "nqI2TokBgL5wWFPZ6Gfu"
  10.       ]
  11.     }
  12.   ]
  13. }

copy

Response fields

FieldTypeDescription
finding1StringThe Id for a first finding in the correlation.
logType1StringThe log type associated with the first finding.
finding2StringThe Id for a second finding in the correlation.
logType2StringThe log type associated with the second finding.
rulesArrayA list of correlation rule IDs associated with the correlated findings.

List correlations for a finding belonging to a log type

You can use the following API to list correlations for certain findings and associated log types:

  1. GET /_plugins/_security_analytics/findings/correlate?finding=425dce0b-f5ee-4889-b0c0-7d15669f0871&detector_type=ad_ldap&nearby_findings=20&time_window=10m

copy

Query parameters

ParameterTypeDescription
findingStringThe finding ID.
detector_typeStringThe log type for the detector.
nearby_findingsNumberThe number of nearby findings with respect to the given finding Id.
time_windowStringSets a time window in which all of the correlations must have occurred together.

Example request

  1. GET /_plugins/_security_analytics/findings/correlate?finding=425dce0b-f5ee-4889-b0c0-7d15669f0871&detector_type=ad_ldap&nearby_findings=20&time_window=10m

copy

Example response

  1. {
  2.   "findings": [
  3.     {
  4.       "finding": "5c661104-aaa9-484b-a91f-9cad4ae6d5f5",
  5.       "detector_type": "others_application",
  6.       "score": 0.000015182109564193524
  7.     },
  8.     {
  9.       "finding": "2485b623-6573-42f4-a055-9b927e38a65f",
  10.       "detector_type": "ad_ldap",
  11.       "score": 0.000001615897872397909
  12.     },
  13.     {
  14.       "finding": "051e00ad-5996-4c41-be20-f992451d1331",
  15.       "detector_type": "windows",
  16.       "score": 0.000016230604160227813
  17.     },
  18.     {
  19.       "finding": "f11ca8a3-50d7-4074-a951-51439aa9e67b",
  20.       "detector_type": "s3",
  21.       "score": 0.000001759401811796124
  22.     },
  23.     {
  24.       "finding": "9b86980e-5fb7-4c5a-bd1b-879a1e3baf12",
  25.       "detector_type": "network",
  26.       "score": 0.0000016306962606904563
  27.     },
  28.     {
  29.       "finding": "e7dea5a1-164f-48f9-880e-4ba33e508713",
  30.       "detector_type": "network",
  31.       "score": 0.00001632626481296029
  32.     }
  33.   ]
  34. }

copy

Response fields

FieldTypeDescription
findingStringThe finding ID.
detector_typeStringThe log type associated with the finding.
scoreNumberThe correlation score for the correlated finding. The score is based on the proximity of relevant findings in the threat scenario defined by the correlation rule.

List correlation alerts

You can use the following API to list correlation alerts:

  1. GET /_plugins/_security_analytics/correlationAlerts

copy

Query parameters

ParameterTypeDescription 
correlation_rule_idStringThe correlation rule ID.Optional

Example request

  1. GET /_plugins/_security_analytics/correlations?correlation_rule_id=VjY0MpABPzR_pcEveVRq

copy

Example response

  1. {
  2.     "correlationAlerts": [
  3.         {
  4.             "correlated_finding_ids": [
  5.                 "4f867df9-c9cb-4dc1-84bb-6c8b575f1a54"
  6.             ],
  7.             "correlation_rule_id": "VjY0MpABPzR_pcEveVRq",
  8.             "correlation_rule_name": "rule-corr",
  9.             "user": null,
  10.             "id": "8532c08b-3ab5-4e95-a1c2-5884c4cd41a5",
  11.             "version": 1,
  12.             "schema_version": 1,
  13.             "trigger_name": "trigger1",
  14.             "state": "ACTIVE",
  15.             "error_message": null,
  16.             "severity": "1",
  17.             "action_execution_results": [],
  18.             "start_time": "2024-06-19T20:37:08.257Z",
  19.             "end_time": "2024-06-19T20:42:08.257Z",
  20.             "acknowledged_time": null
  21.         },
  22.         {
  23.             "correlated_finding_ids": [
  24.                 "30d2109f-76bb-44ad-8f68-6daa905e018d"
  25.             ],
  26.             "correlation_rule_id": "VjY0MpABPzR_pcEveVRq",
  27.             "correlation_rule_name": "rule-corr",
  28.             "user": null,
  29.             "id": "8bba85d9-a7fc-4c87-b35e-a7236b87159f",
  30.             "version": 1,
  31.             "schema_version": 1,
  32.             "trigger_name": "trigger1",
  33.             "state": "ACTIVE",
  34.             "error_message": null,
  35.             "severity": "1",
  36.             "action_execution_results": [],
  37.             "start_time": "2024-06-19T20:43:08.208Z",
  38.             "end_time": "2024-06-19T20:48:08.208Z",
  39.             "acknowledged_time": null
  40.         }
  41.     ],
  42.     "total_alerts": 2
  43. }

copy

Acknowledge correlation alerts

You can use the following API to acknowledge the correlation alerts.

Example request

  1. POST /_plugins/_security_analytics/_acknowledge/correlationAlerts
  2. {
  3.    "alertIds": ["8532c08b-3ab5-4e95-a1c2-5884c4cd41a5", "8bba85d9-a7fc-4c87-b35e-a7236b87159f"]
  4. }

copy

Example response

  1. {
  2.     "acknowledged": [
  3.         {
  4.             "correlated_finding_ids": [
  5.                 "4f867df9-c9cb-4dc1-84bb-6c8b575f1a54"
  6.             ],
  7.             "correlation_rule_id": "VjY0MpABPzR_pcEveVRq",
  8.             "correlation_rule_name": "rule-corr",
  9.             "user": null,
  10.             "id": "8532c08b-3ab5-4e95-a1c2-5884c4cd41a5",
  11.             "version": 1,
  12.             "schema_version": 1,
  13.             "trigger_name": "trigger1",
  14.             "state": "ACTIVE",
  15.             "error_message": null,
  16.             "severity": "1",
  17.             "action_execution_results": [],
  18.             "start_time": "2024-06-19T20:37:08.257Z",
  19.             "end_time": "2024-06-19T20:42:08.257Z",
  20.             "acknowledged_time": null
  21.         },
  22.         {
  23.             "correlated_finding_ids": [
  24.                 "30d2109f-76bb-44ad-8f68-6daa905e018d"
  25.             ],
  26.             "correlation_rule_id": "VjY0MpABPzR_pcEveVRq",
  27.             "correlation_rule_name": "rule-corr",
  28.             "user": null,
  29.             "id": "8bba85d9-a7fc-4c87-b35e-a7236b87159f",
  30.             "version": 1,
  31.             "schema_version": 1,
  32.             "trigger_name": "trigger1",
  33.             "state": "ACTIVE",
  34.             "error_message": null,
  35.             "severity": "1",
  36.             "action_execution_results": [],
  37.             "start_time": "2024-06-19T20:43:08.208Z",
  38.             "end_time": "2024-06-19T20:48:08.208Z",
  39.             "acknowledged_time": null
  40.         }
  41.     ],
  42.     "failed": []
  43. }

copy