TorkTORK

Core Features

Circuit Breaker

Automatically stop runaway agents and prevent cascading failures with intelligent circuit breakers.

How It Works

The circuit breaker monitors agent behavior and automatically trips when anomalies are detected, preventing runaway costs, infinite loops, or cascading failures.

CLOSED

Normal operation, requests flow through

OPEN

Tripped, all requests blocked

HALF-OPEN

Testing recovery, limited requests

Configuration

Configure circuit breaker thresholds per agent or globally:

python
from tork_governance import TorkClient

client = TorkClient()

# Configure circuit breaker for an agent
client.circuit_breaker.configure(
    agent_id="agent-123",
    failure_threshold=5,        # Trips after 5 failures
    success_threshold=3,        # Closes after 3 successes in half-open
    timeout_seconds=60,         # Time before attempting recovery
    failure_rate_threshold=0.5, # Trips if >50% requests fail
    slow_call_threshold=5.0,    # Seconds to consider a call "slow"
    slow_call_rate=0.8,         # Trips if >80% calls are slow
)

Trip Triggers

Circuit breakers can trip based on multiple conditions:

Error Rate

Too many consecutive failures or high failure percentage

Latency

Response times consistently exceeding thresholds

Cost Anomaly

Spending rate significantly above baseline

Token Usage

Token consumption exceeding limits

Loop Detection

Repetitive patterns indicating infinite loops

Checking Circuit Status

python
# Check circuit breaker status
status = client.circuit_breaker.status(agent_id="agent-123")

print(f"State: {status.state}")          # CLOSED, OPEN, or HALF_OPEN
print(f"Failure count: {status.failure_count}")
print(f"Last failure: {status.last_failure_time}")

# Manual controls
client.circuit_breaker.trip(agent_id="agent-123", reason="Manual intervention")
client.circuit_breaker.reset(agent_id="agent-123")

Webhook Notifications

Receive notifications when circuit breakers change state:

json
{
  "event": "circuit_breaker.opened",
  "agent_id": "agent-123",
  "timestamp": "2024-01-15T10:30:00Z",
  "data": {
    "previous_state": "CLOSED",
    "new_state": "OPEN",
    "trigger": "failure_rate",
    "failure_rate": 0.65,
    "failure_count": 13,
    "window_duration": "60s"
  }
}

Important: When a circuit breaker trips, all requests to that agent are immediately blocked until manual reset or automatic recovery.

Documentation

Learn to integrate TORK

Upgrade Plan

Current: free

Support

Get help from our team