Arato Build GitHub Action is available on the GitHub Marketplace.
This powerful integration enables teams to automate and safeguard their generative AI experiment builds as part of their GitHub CI pipelines.
By combining Arato’s GenAI evaluation capabilities with your existing development workflow, you can ensure that only high-quality, reliable changes make it into production.
What Is the Arato Build GitHub Action?
The Arato Build GitHub Action is a CI/CD integration that runs your Arato GenAI experiments during a GitHub workflow. It runs your evals and key performance metrics (such as cost, latency, and semantic similarity) and automatically flags changes that fall below your configured thresholds.
It’s built for teams who want to:
Automate GenAI experiment testing as part of PR checks or deployments.
Enforce quality gates before GenAI changes are merged.
Continuously monitor model behavior in evolving codebases.
How It Works
Define your notebook in Arato.
Add the GitHub Action to your repo’s workflow.
Configure evaluation thresholds and API keys using GitHub Secrets.
Run builds automatically or manually, and let Arato validate your experiment’s behavior.
Quick Setup Guide
Simply add this workflow to .github/workflows/arato-build.yml:
name: Arato Build
on:
workflow_dispatch:
inputs:
experiments:
description: 'Experiments to build (e.g., my-notebook)'
required: true
push:
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Arato Build
uses: AratoAi/[email protected]
with:
experiments: ${{ inputs.experiments }}
api_keys: |
{
"openai_api_key": "${{ secrets.OPENAI_API_KEY }}",
"anthropic_api_key": "${{ secrets.ANTHROPIC_API_KEY }}"
}
arato_api_key: ${{ secrets.ARATO_API_KEY }}
threshold: |
{
"eval_pass_rate": 0.8,
"cost": 0.01,
"latency": 3000,
"tokens": 100,
"semantic_similarity": 0.7
}
You can trigger builds:
Automatically on every push.
Manually through GitHub → Actions → Arato Build → Run workflow → enter your notebook ID.
Learn More - View the Action on GitHub Marketplace