Evaluate trails with OPA policies

Learn how to write safe OPA/Rego policies for kosli evaluate trail and kosli evaluate trails, including design rules that prevent false-positive compliance results.

The kosli evaluate commands let you evaluate Kosli trails against custom policies written in Rego. This is useful for enforcing rules like “every artifact must have an approved pull request” or “all security scans must pass”, and for gating deployments in CI/CD pipelines based on those rules.

In this tutorial, you’ll write and evaluate policies against real trails in public Kosli orgs. Along the way, you’ll learn three design rules that prevent a Rego policy from granting false-positive compliance results.

1. Prerequisites

   To follow this tutorial, you need to:

   • Install Kosli CLI.

   • Get a Kosli API token.

   • Set the KOSLI_API_TOKEN environment variable to your token:

     export KOSLI_API_TOKEN=<your-api-token>
     

   You don’t need OPA installed — the Kosli CLI has a built-in Rego evaluator. You just need to write a .rego policy file.

2. Write a policy

   Create a file called pr-approved.rego with the following content:

   pr-approved.rego

   package policy
   
   import rego.v1
   
   pr_attestation_name := data.params.pr_attestation_name
   
   default allow = false
   
   violations contains msg if {
       some trail in input.trails
       some pr in trail.compliance_status.attestations_statuses[pr_attestation_name].pull_requests
       count(pr.approvers) == 0
       msg := sprintf("trail '%v': pull-request %v has no approvers", [trail.name, pr.url])
   }
   
   trail_is_approved(trail) if {
       every pr in trail.compliance_status.attestations_statuses[pr_attestation_name].pull_requests {
           count(pr.approvers) > 0
       }
   }
   
   allow if {
       every trail in input.trails {
           trail_is_approved(trail)
       }
   }
   

   This policy applies three design rules that every evaluate policy should follow.

   Rule 1: default allow = false — fail safe

   Trails are denied unless the policy explicitly allows them. Anything the policy cannot positively verify is treated as non-compliant. This matches Kosli’s compliance direction: a false non-compliant blocks a good trail (recoverable); a false compliant passes a bad one (not recoverable).

   The alias pr_attestation_name := data.params.pr_attestation_name reads the attestation name from a params file rather than hardcoding it. Different orgs and flows use different names for their pull-request attestation (for example "pull-request" or "pr"). If the param is absent, pr_attestation_name is undefined, the lookup into attestations_statuses fails, trail_is_approved does not fire, and allow stays false — the correct fail-safe.

   Rule 2: Drive allow via a positive assertion, not the absence of violations

   allow fires through trail_is_approved, which makes a positive claim: every PR has at least one approver. It is never driven by count(violations) == 0.

   The following pattern looks equivalent but is not safe:

   # unsafe
   allow if {
       count(violations) == 0
   }
   

   If the violations rule body references a field that does not exist — a typo in pull_requests, an unexpected schema change, a missing key — the rule body silently produces no messages. The violations set is empty, count(violations) == 0 is true, and allow fires even though no PRs were actually checked. The trail receives a false-positive compliant result.

   With the safe pattern, if pull_requests is undefined, every pr in ... fails to evaluate, trail_is_approved does not fire, and allow stays false.

   Rule 3: Violations provide diagnostics only

   violations explains why a policy was denied — it does not decide whether it was denied. When a violations rule body encounters an undefined reference, it silently produces no message. This is the safe failure mode: you lose a diagnostic, not a compliance check.

   See the Rego Policy reference for the full policy contract, input data shape, and exit code behaviour.

3. Evaluate multiple trails

   Let’s evaluate several trails from the public cyber-dojo org against our policy. The kosli evaluate trails command fetches trail data from Kosli and passes it to the policy as input.trails:

   kosli evaluate trails \
     --policy pr-approved.rego \
     --params '{"pr_attestation_name": "pull-request"}' \
     --org cyber-dojo \
     --flow dashboard-ci \
     9978a1ca82c273a68afaa85fc37dd60d1e394f84 \
     b334d371eb85c9a5c811776de1b65fb80b52d952 \
     5abd63aa1d64af7be5b5900af974dc73ae425bd6 \
     cb3ec71f5ce1103779009abaf4e8f8a3ed97d813
   

   The cyber-dojo project doesn’t require PR approvals, so you’ll see violations:

   RESULT:      DENIED
   VIOLATIONS:  trail '5abd63aa1d64af7be5b5900af974dc73ae425bd6': pull-request https://github.com/cyber-dojo/dashboard/pull/342 has no approvers
                trail '9978a1ca82c273a68afaa85fc37dd60d1e394f84': pull-request https://github.com/cyber-dojo/dashboard/pull/344 has no approvers
                trail 'b334d371eb85c9a5c811776de1b65fb80b52d952': pull-request https://github.com/cyber-dojo/dashboard/pull/343 has no approvers
                trail 'cb3ec71f5ce1103779009abaf4e8f8a3ed97d813': pull-request https://github.com/cyber-dojo/dashboard/pull/341 has no approvers
   

   Now try the kosli-public org, where PRs do have approvers. This org names the attestation "pr":

   kosli evaluate trails \
     --policy pr-approved.rego \
     --params '{"pr_attestation_name": "pr"}' \
     --org kosli-public \
     --flow cli \
     5a0f3c0 \
     167ed93 \
     030cc31
   

   RESULT:  ALLOWED
   
4. Evaluate a single trail

   The kosli evaluate trail (singular) command evaluates facts within a single trail. For example, you might check that a Snyk container scan found no high-severity vulnerabilities.

   Save this as snyk-no-high-vulns.rego:

   snyk-no-high-vulns.rego

   package policy
   
   import rego.v1
   
   default allow = false
   
   artifact_scan_is_clean(artifact) if {
       snyk := artifact.attestations_statuses["snyk-container-scan"]
       every result in snyk.processed_snyk_results.results {
           result.high_count == 0
       }
   }
   
   allow if {
       every _, artifact in input.trail.compliance_status.artifacts_statuses {
           artifact_scan_is_clean(artifact)
       }
   }
   
   violations contains msg if {
       some name, artifact in input.trail.compliance_status.artifacts_statuses
       snyk := artifact.attestations_statuses["snyk-container-scan"]
       some result in snyk.processed_snyk_results.results
       result.high_count > 0
       msg := sprintf("artifact '%v': snyk container scan found %d high severity vulnerabilities", [name, result.high_count])
   }
   

   allow fires only when artifact_scan_is_clean succeeds for every artifact. If snyk-container-scan is absent or processed_snyk_results is undefined, artifact_scan_is_clean fails to fire and allow stays false.

   Use --attestations to enrich only the snyk data (faster than fetching all attestation details). The value uses the format artifact-name.attestation-type. Here, dashboard is the artifact name and snyk-container-scan is the attestation name:

   kosli evaluate trail \
     --policy snyk-no-high-vulns.rego \
     --org cyber-dojo \
     --flow dashboard-ci \
     --attestations dashboard.snyk-container-scan \
     44ca5fa2630947cf375fdbda10972a4bedaaaba3
   

   RESULT:  ALLOWED
   

   The trail has zero high-severity vulnerabilities, so the policy allows it.

   The input.trail / input.trails distinction and the full input data shape are documented in the Rego Policy reference.

5. Pass parameters to a policy

   Policies often need thresholds that vary by environment — stricter in production than in staging. Use the --params flag to pass these values as data.params rather than hardcoding them in the policy.

   Save this as snyk-severity-threshold.rego:

   snyk-severity-threshold.rego

   package policy
   
   import rego.v1
   
   max_high   := data.params.max_high
   max_medium := data.params.max_medium
   
   default allow = false
   
   artifact_within_threshold(artifact) if {
       snyk := artifact.attestations_statuses["snyk-container-scan"]
       every result in snyk.processed_snyk_results.results {
           result.high_count   <= max_high
           result.medium_count <= max_medium
       }
   }
   
   allow if {
       every _, artifact in input.trail.compliance_status.artifacts_statuses {
           artifact_within_threshold(artifact)
       }
   }
   
   violations contains msg if {
       some name, artifact in input.trail.compliance_status.artifacts_statuses
       snyk := artifact.attestations_statuses["snyk-container-scan"]
       some result in snyk.processed_snyk_results.results
       result.high_count > max_high
       msg := sprintf("artifact '%v': %d high-severity vulnerabilities exceed limit of %d", [name, result.high_count, max_high])
   }
   
   violations contains msg if {
       some name, artifact in input.trail.compliance_status.artifacts_statuses
       snyk := artifact.attestations_statuses["snyk-container-scan"]
       some result in snyk.processed_snyk_results.results
       result.medium_count > max_medium
       msg := sprintf("artifact '%v': %d medium-severity vulnerabilities exceed limit of %d", [name, result.medium_count, max_medium])
   }
   

   Why the aliases at the top matter

   max_high := data.params.max_high is not just shorthand. In the compliance path, result.high_count <= max_high is a positive bound check. If max_high is absent from the params file, this condition is undefined, artifact_within_threshold fails to fire, and allow stays false. That is the correct fail-safe behaviour.

   Compare this to a policy that drives allow through the absence of violations:

   # unsafe
   violations contains msg if {
       result.high_count > data.params.max_high  # fails silently if max_high is absent
       msg := ...
   }
   
   allow if { count(violations) == 0 }
   

   If data.params.max_high is absent, the violations rule body fails silently, the set stays empty, and allow fires. A misconfigured params file grants compliance rather than denying it.

   Testing with kosli evaluate input

   You can test a policy locally without a real trail using kosli evaluate input. Create a minimal input file:

   cat > scan-input.json << 'EOF'
   {
     "trail": {
       "compliance_status": {
         "artifacts_statuses": {
           "dashboard": {
             "attestations_statuses": {
               "snyk-container-scan": {
                 "processed_snyk_results": {
                   "results": [{"high_count": 2, "medium_count": 5}]
                 }
               }
             }
           }
         }
       }
     }
   }
   EOF
   

   Evaluate with permissive staging thresholds:

   kosli evaluate input \
     --input-file scan-input.json \
     --policy snyk-severity-threshold.rego \
     --params '{"max_high": 5, "max_medium": 10}'
   

   RESULT:  ALLOWED
   

   Apply stricter production thresholds:

   kosli evaluate input \
     --input-file scan-input.json \
     --policy snyk-severity-threshold.rego \
     --params '{"max_high": 0, "max_medium": 3}'
   

   RESULT:      DENIED
   VIOLATIONS:  artifact 'dashboard': 2 high-severity vulnerabilities exceed limit of 0
                artifact 'dashboard': 5 medium-severity vulnerabilities exceed limit of 3
   

   Now verify the fail-safe: omit max_high from params entirely:

   kosli evaluate input \
     --input-file scan-input.json \
     --policy snyk-severity-threshold.rego \
     --params '{"max_medium": 10}'
   

   RESULT:  DENIED
   

   allow is false even though no violation message was produced. The missing param causes the compliance check to fail, not to vacuously pass. Always verify this explicitly when writing a policy that relies on params.

   You can also load parameters from a file using the @ prefix:

   echo '{"max_high": 0, "max_medium": 3}' > params-prod.json
   
   kosli evaluate input \
     --input-file scan-input.json \
     --policy snyk-severity-threshold.rego \
     --params @params-prod.json
   

   The --params flag works the same way on kosli evaluate trail and kosli evaluate trails — parameters are always available as data.params in the policy.

6. Explore the policy input with --show-input

   When writing policies, it helps to see exactly what data is available. Use --show-input combined with --output json to see the full input that gets passed to the policy:

   kosli evaluate trail \
     --policy snyk-no-high-vulns.rego \
     --org cyber-dojo \
     --flow dashboard-ci \
     --attestations dashboard.snyk-container-scan \
     --show-input \
     --output json \
     44ca5fa2630947cf375fdbda10972a4bedaaaba3
   

   This outputs the evaluation result along with the complete input object. You can pipe it through jq to explore the structure:

   kosli evaluate trail \
     --policy snyk-no-high-vulns.rego \
     --org cyber-dojo \
     --flow dashboard-ci \
     --attestations dashboard.snyk-container-scan \
     --show-input \
     --output json \
     44ca5fa2630947cf375fdbda10972a4bedaaaba3 2>/dev/null | jq '.input.trail.compliance_status | keys'
   

   [
     "artifacts_statuses",
     "attestations_statuses",
     "evaluated_at",
     "flow_template_id",
     "is_compliant",
     "status"
   ]
   

   Use the --attestations flag to limit which attestations are enriched with full detail. The flag filters by attestation name (not type). For example, --attestations pull-request fetches only details for attestations named pull-request, which speeds up evaluation and reduces noise when exploring the input.

7. Use in CI/CD

   The kosli evaluate commands exit with 0 on allow and 1 on deny or error — making them straightforward to use as pipeline gates. See the Rego Policy reference for details on distinguishing denial from command failure.

   # Example: gate a deployment on policy evaluation
   if kosli evaluate trail \
     --policy policies/pr-approved.rego \
     --org "$KOSLI_ORG" \
     --flow "$FLOW_NAME" \
     "$GIT_COMMIT"; then
     echo "Policy passed -- proceeding with deployment"
     # ... deploy commands ...
   else
     echo "Policy denied -- blocking deployment"
     exit 1
   fi
   

   This pattern lets you enforce custom compliance rules as part of your delivery pipeline, using the same trail data that Kosli already collects.

8. Record the evaluation

   After evaluating a trail, you can record the result as an attestation. This creates an audit record in Kosli that captures the policy, the full evaluation report, and any violations.

   This step requires write access to your Kosli org. The examples below use variables you’d set in your CI/CD pipeline. In your own pipeline you’d use your own policy file — here we use my-policy.rego as a placeholder:

   # Run the evaluation and save the full JSON report to a file
   # (|| true prevents the step from failing when the policy denies)
   kosli evaluate trail "$TRAIL_NAME" \
     --policy my-policy.rego \
     --org "$KOSLI_ORG" \
     --flow "$FLOW_NAME" \
     --show-input \
     --output json > eval-report.json 2>/dev/null || true
   
   # Read the allow/deny result from the report
   is_compliant=$(jq --raw-output '.allow' eval-report.json)
   
   # Extract violations as structured user-data
   jq '{violations: .violations}' eval-report.json > eval-violations.json
   
   # Attest the result
   kosli attest generic \
     --name opa-evaluation \
     --flow "$FLOW_NAME" \
     --trail "$TRAIL_NAME" \
     --org "$KOSLI_ORG" \
     --compliant="$is_compliant" \
     --attachments my-policy.rego,eval-report.json \
     --user-data eval-violations.json
   

   This creates a generic attestation on the trail with:

   • --compliant set based on whether the policy allowed or denied — read directly from the JSON report rather than relying on the exit code, which avoids issues with set -e in CI environments like GitHub Actions
   • --attachments containing the Rego policy (for reproducibility) and the full JSON evaluation report (including the input data the policy evaluated)
   • --user-data containing the violations, which appear in the Kosli UI as structured metadata on the attestation

   Use --compliant=value (with =) not --compliant value (with a space). Boolean flags in Kosli CLI require the = syntax when passing false — otherwise false is interpreted as a positional argument.

What you’ve accomplished

You have written OPA/Rego policies using the three design rules that prevent false-positive compliance results: fail-safe default, compliance via positive assertion, and violations as diagnostics only. You’ve evaluated Kosli trails against those policies, tested safety properties locally with kosli evaluate input, and recorded evaluation results as attestations.

From here you can:

• Explore evaluated trails in the Kosli app
• Gate deployments in CI/CD pipelines using kosli evaluate trail exit codes
• Use environment-specific params files to enforce different thresholds per environment
• Extend your policies to check other attestation types. See kosli evaluate trail and kosli evaluate trails for the full flag reference