GitHub Action Integration¶

Zero-config evidence verification for AI agents. Native GitHub Security tab integration.

GitHub Marketplace: assay-ai-agent-security

Canonical public action slug: Rul1an/assay-action@v2. Monorepo-internal workflow tests may use ./assay-action; that local path is not the recommended external reference.

30-Second Setup¶

# .github/workflows/assay.yaml
name: Evidence Verification
on: [push, pull_request]

permissions:
  contents: read
  security-events: write

jobs:
  assay:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: Rul1an/assay-action@v2

Copy, paste, done. The action auto-discovers evidence bundles and reports findings to the Security tab.

Quick Start¶

- uses: Rul1an/assay-action@v2

Zero config. Discovers evidence bundles, verifies integrity, uploads SARIF.

What It Does¶

Discovers evidence bundles in your repo (.assay/evidence/*.tar.gz)
Verifies bundle integrity (content-addressed IDs)
Lints for security issues with optional compliance packs
Reports to GitHub Security tab + PR comments

Full Workflow Example¶

# .github/workflows/assay.yaml
name: Evidence Verification

on:
  push:
    branches: [main]
  pull_request:

permissions:
  contents: read
  security-events: write
  pull-requests: write

jobs:
  assay:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Run tests with evidence collection
        run: |
          curl -fsSL https://getassay.dev/install.sh | sh
          echo "$HOME/.local/bin" >> $GITHUB_PATH
          assay ci --config eval.yaml --trace-file traces/golden.jsonl --strict --junit .assay/reports/junit.xml --sarif .assay/reports/sarif.json

      - name: Verify evidence
        uses: Rul1an/assay-action@v2
        with:
          fail_on: error
          baseline_key: ${{ github.event.repository.name }}
          write_baseline: ${{ github.ref == format('refs/heads/{0}', github.event.repository.default_branch) }}

Inputs¶

Core Inputs (v2.0)¶

Input	Default	Description
`bundles`	Auto-detect	Glob pattern for evidence bundles
`fail_on`	`error`	Fail threshold: `error`, `warn`, `info`, `none`
`sarif`	`true`	Upload to GitHub Security tab
`comment_diff`	`true`	Post PR comment (only if findings)
`baseline_key`	-	Key for baseline comparison
`write_baseline`	`false`	Save baseline (default branch only)
`version`	`latest`	Assay CLI version

Compliance Pack Input (v2.1)¶

Input	Default	Description
`pack`	-	Compliance pack(s): `eu-ai-act-baseline`, `soc2-baseline`, `./custom.yaml`

BYOS Input (v2.1)¶

Input	Default	Description
`store`	-	BYOS URL: `s3://bucket/prefix`, `gs://bucket`, `az://container`
`store_provider`	`auto`	`aws`, `gcp`, `azure`, or `auto`
`store_role`	-	IAM role/identity for OIDC

Attestation Input (v2.1)¶

Input	Default	Description
`attest`	`false`	Generate artifact attestation

Outputs¶

Output	Description
`verified`	`true` if all bundles verified
`findings_error`	Error count
`findings_warn`	Warning count
`reports_dir`	Reports directory path
`pack_applied`	Applied pack IDs (v2.1)
`resolved_pack_refs`	Resolved pack refs as `name@version` (v2.1)
`pack_score`	Compliance score 0-100 (v2.1)
`pack_status`	Pack status: `disabled`, `success`, `missing_pack`, `invalid_pack`, `runtime_error` (v2.1)
`pack_error_kind`	Pack error category when pack status is non-success (v2.1)
`bundle_url`	BYOS bundle URL (v2.1)
`attestation_id`	Attestation UUID (v2.1)

Permission Model¶

# Minimal (lint only)
permissions:
  contents: read

# With SARIF upload
permissions:
  contents: read
  security-events: write

# With PR comments
permissions:
  contents: read
  security-events: write
  pull-requests: write

# With attestation + OIDC (v2.1)
permissions:
  contents: read
  security-events: write
  attestations: write
  id-token: write

Compliance Packs¶

Lint evidence against regulatory requirements:

- uses: Rul1an/assay-action@v2
  with:
    pack: eu-ai-act-baseline

SARIF output includes article references (Article 12(1), etc.) for audit trails.

Available packs:

Pack	Coverage
`eu-ai-act-baseline`	Article 12 logging requirements
`soc2-baseline`	Control mapping (coming soon)

Custom packs:

- uses: Rul1an/assay-action@v2
  with:
    pack: ./my-org-rules.yaml

BYOS Push with OIDC¶

Push evidence to your own storage. No static credentials.

AWS S3¶

permissions:
  id-token: write
  contents: read

jobs:
  assay:
    runs-on: ubuntu-latest
    steps:
      - uses: Rul1an/assay-action@v2
        with:
          store: s3://my-bucket/evidence
          store_provider: aws
          store_role: arn:aws:iam::123456789:role/assay-evidence-push

Requires IAM trust policy for token.actions.githubusercontent.com.

GCP Cloud Storage¶

- uses: Rul1an/assay-action@v2
  with:
    store: gs://my-bucket/evidence
    store_provider: gcp
    store_role: projects/my-project/locations/global/workloadIdentityPools/github/providers/github

Artifact Attestation¶

Generate SLSA-aligned provenance for evidence bundles:

permissions:
  attestations: write
  id-token: write

steps:
  - uses: Rul1an/assay-action@v2
    with:
      attest: true

Verify locally:

gh attestation verify bundle.tar.gz --owner Rul1an

Attestations only run on push to default branch (security).

Common Patterns (Copy-Paste)¶

Pattern 1: CI Gate with Baseline¶

Block PRs that introduce new security findings:

name: Security Gate
on: [push, pull_request]

permissions:
  contents: read
  security-events: write
  pull-requests: write

jobs:
  assay:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Run tests
        run: pytest tests/

      - uses: Rul1an/assay-action@v2
        with:
          fail_on: error
          baseline_key: ${{ github.event.repository.name }}
          write_baseline: ${{ github.ref == format('refs/heads/{0}', github.event.repository.default_branch) }}

Pattern 2: Compliance Reporting¶

EU AI Act Article 12 compliance checks with SARIF output:

- uses: Rul1an/assay-action@v2
  with:
    pack: eu-ai-act-baseline
    fail_on: warn  # Fail on warnings too

Pattern 3: Enterprise Pipeline (Full v2.1)¶

Complete pipeline with BYOS, attestation, and compliance:

name: Evidence Pipeline
on:
  push:
    branches: [main]
  pull_request:

permissions:
  contents: read
  security-events: write
  pull-requests: write
  attestations: write
  id-token: write

jobs:
  assay:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Run tests
        run: |
          curl -fsSL https://getassay.dev/install.sh | sh
          assay ci --config eval.yaml --trace-file traces/golden.jsonl --strict --junit .assay/reports/junit.xml --sarif .assay/reports/sarif.json

      - uses: Rul1an/assay-action@v2
        with:
          pack: eu-ai-act-baseline
          store: s3://my-bucket/evidence
          store_role: arn:aws:iam::123456789:role/AssayRole
          attest: true
          baseline_key: main
          write_baseline: ${{ github.ref == format('refs/heads/{0}', github.event.repository.default_branch) }}

Examples¶

Baseline Comparison¶

Detect regressions against your default branch:

- uses: Rul1an/assay-action@v2
  with:
    baseline_key: unit-tests
    write_baseline: ${{ github.ref == format('refs/heads/{0}', github.event.repository.default_branch) }}

Matrix Builds¶

Multiple test suites:

jobs:
  test:
    strategy:
      matrix:
        suite: [unit, integration, e2e]
    steps:
      - uses: actions/checkout@v4

      - name: Run ${{ matrix.suite }} tests
        run: assay ci --config eval.yaml --trace-file traces/${{ matrix.suite }}.jsonl --strict --junit .assay/reports/${{ matrix.suite }}.junit.xml --sarif .assay/reports/${{ matrix.suite }}.sarif.json

      - uses: Rul1an/assay-action@v2
        with:
          bundles: '.assay/evidence/${{ matrix.suite }}/*.tar.gz'

Full v2.1 Workflow¶

name: Evidence Pipeline

on:
  push:
    branches: [main]
  pull_request:

permissions:
  contents: read
  security-events: write
  pull-requests: write
  attestations: write
  id-token: write

jobs:
  assay:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Run tests
        run: |
          curl -fsSL https://getassay.dev/install.sh | sh
          assay ci --config eval.yaml --trace-file traces/golden.jsonl --strict --junit .assay/reports/junit.xml --sarif .assay/reports/sarif.json

      - name: Verify with compliance pack
        uses: Rul1an/assay-action@v2
        with:
          pack: eu-ai-act-baseline
          store: s3://my-bucket/evidence
          store_provider: aws
          store_role: ${{ secrets.AWS_ROLE_ARN }}
          attest: true
          baseline_key: main
          write_baseline: ${{ github.ref == format('refs/heads/{0}', github.event.repository.default_branch) }}

Manual CLI Workflow¶

Using the CLI directly:

jobs:
  assay:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      security-events: write

    steps:
      - uses: actions/checkout@v4

      - name: Install Assay
        run: |
          curl -fsSL https://getassay.dev/install.sh | sh
          echo "$HOME/.local/bin" >> $GITHUB_PATH

      - name: Run tests
        run: assay ci --config eval.yaml --trace-file traces/golden.jsonl --strict --junit .assay/reports/junit.xml --sarif .assay/reports/sarif.json

      - name: Export evidence
        run: assay evidence export --profile assay-profile.yaml --out evidence.tar.gz

      - name: Lint with pack
        run: assay evidence lint evidence.tar.gz --pack eu-ai-act-baseline --format sarif > results.sarif
        continue-on-error: true

      # Assay writes reports (JUnit/SARIF) as "Best Effort".
      # Failures to write reports (e.g. bad path) print a WARNING but do not fail the step.
      # The primary artifact 'run.json' is also written on a "Best Effort" basis.

      - name: Upload SARIF
        uses: github/codeql-action/upload-sarif@v4
        if: always()
        with:
          sarif_file: results.sarif

Troubleshooting¶

No evidence bundles found¶

The action looks for: - .assay/evidence/*.tar.gz - evidence/*.tar.gz

Generate with:

assay ci --config eval.yaml --trace-file traces/golden.jsonl --strict --junit .assay/reports/junit.xml --sarif .assay/reports/sarif.json

SARIF upload fails¶

Check security-events: write permission.

PR comments not appearing¶

Check pull-requests: write permission and that the action runs on a pull_request event.

OIDC authentication fails¶

Verify IAM trust relationship includes your repo:

{
  "Condition": {
    "StringLike": {
      "token.actions.githubusercontent.com:sub": "repo:YOUR-ORG/YOUR-REPO:*"
    }
  }
}

Compliance pack step fails¶

Check action outputs: - pack_status=missing_pack: pack ref not found (typo or missing file path) - pack_status=invalid_pack: pack schema/validation error - pack_status=runtime_error: unexpected lint/runtime failure

The action also exposes pack_error_kind for machine-readable CI triage.

Behavior note: - pack loading/validation/runtime errors fail the action step - policy findings continue to follow the normal fail_on threshold

Security Notes¶

Write operations (baseline, BYOS push, attestation) only run on push to default branch
Fork PRs cannot trigger write operations (GitHub Actions security model)
BYOS push requires OIDC trust relationship configured in cloud IAM