Introduction

Integrate with Aliyun content security service for detections of input and output of LLMs, ensuring that application content is legal and compliant.

Runtime Properties

Plugin Phase: CUSTOM Plugin Priority: 300

Configuration

NameTypeRequirementDefaultDescription
serviceNamestringrequried-service name
servicePortstringrequried-service port
serviceHoststringrequried-Host of Aliyun content security service endpoint
accessKeystringrequried-Aliyun accesskey
secretKeystringrequried-Aliyun secretkey
checkRequestbooloptionalfalsecheck if the input is legal
checkResponsebooloptionalfalsecheck if the output is legal
requestCheckServicestringoptionalllm_query_moderationAliyun yundun service name for input check
responseCheckServicestringoptionalllm_response_moderationAliyun yundun service name for output check
requestContentJsonPathstringoptionalmessages.@reverse.0.contentSpecify the jsonpath of the content to be detected in the request body
responseContentJsonPathstringoptionalchoices.0.message.contentSpecify the jsonpath of the content to be detected in the response body
responseStreamContentJsonPathstringoptionalchoices.0.delta.contentSpecify the jsonpath of the content to be detected in the streaming response body
denyCodeintoptional200Response status code when the specified content is illegal
denyMessagestringoptionalDrainage/non-streaming response in openai format, the answer content is the suggested answer from Alibaba Cloud content security
Response content when the specified content is illegal

Examples of configuration

  1. serviceName: safecheck.dns
  2. servicePort: 443
  3. serviceHost: “green-cip.cn-shanghai.aliyuncs.com”
  4. accessKey: “XXXXXXXXX”
  5. secretKey: “XXXXXXXXXXXXXXX”
  6. checkRequest: true
  1. serviceName: safecheck.dns
  2. servicePort: 443
  3. serviceHost: green-cip.cn-shanghai.aliyuncs.com
  4. accessKey: “XXXXXXXXX”
  5. secretKey: “XXXXXXXXXXXXXXX”
  6. checkRequest: true
  7. checkResponse: true

Observability

Metric

ai-security-guard plugin provides following metrics:

  • ai_sec_request_deny: count of requests denied at request phase
  • ai_sec_response_deny: count of requests denied at response phase

Trace

ai-security-guard plugin provides following span attributes:

  • ai_sec_risklabel: risk type of this request
  • ai_sec_deny_phase: denied phase of this request, value can be request/response