treeherder（treeherder 搜索）

Treeherder

Query Mozilla Treeherder for CI job results, failure analysis, and performance data.

Tools

This skill uses two CLI tools:

Use treeherder-cli for revision-based failure analysis, comparing revisions, test history, log searching, and artifact downloads.

Use lumberjackth for listing pushes, querying failures by bug ID, viewing error lines with bug suggestions, performance alerts, and filtering by result/state/tier.

Quick Start

treeherder-cli (primary)

# Get failed jobs for a revision
treeherder-cli a13b9fc22101 --json

# Filter by job name or platform
treeherder-cli a13b9fc22101 --filter "mochitest" --json
treeherder-cli a13b9fc22101 --platform "linux.*64" --json

# Group failures by test (cross-platform view)
treeherder-cli a13b9fc22101 --group-by test --json

# Compare revisions to find regressions
treeherder-cli a13b9fc22101 --compare b2c3d4e5f678 --json

# Check test history for intermittent detection
treeherder-cli --history "test_audio_playback" --history-count 10 --repo try --json

# Show similar job history for a failed job ID
treeherder-cli --similar-history 543981186 --similar-count 100 --repo try --json

# Fetch logs with pattern matching
treeherder-cli a13b9fc22101 --fetch-logs --pattern "ASSERTION|CRASH" --json

# Download artifacts
treeherder-cli a13b9fc22101 --download-artifacts --artifact-pattern "screenshot|errorsummary"

# Watch mode with notification
treeherder-cli a13b9fc22101 --watch --notify

# Switch repository (default: autoland)
treeherder-cli a13b9fc22101 --repo try --json

lumberjackth (secondary)

# List recent pushes
uvx --from lumberjackth lj pushes autoland -n 10

# Get jobs for a push with result/tier filtering
uvx --from lumberjackth lj jobs autoland --push-id 12345 --result testfailed --tier 1

# Watch jobs with auto-refresh
uvx --from lumberjackth lj jobs try -r abc123 -w -i 60

# Query failures by bug ID
uvx --from lumberjackth lj failures 2012615 -t autoland -p "windows.*24h2"

# Show errors and bug suggestions
uvx --from lumberjackth lj errors autoland 545896732

# Performance alerts
uvx --from lumberjackth lj perf-alerts -r autoland -n 10

# JSON output
uvx --from lumberjackth lj --json jobs autoland --push-id 12345

When to Use Which Tool

Bug Suggestions for Failing Jobs

Neither CLI tool supports bulk querying of known intermittent bugs for failing jobs. Use the Treeherder REST API directly to check whether test failures have associated Bugzilla bugs.

Workflow

1. Get failing job IDs from treeherder-cli JSON output
2. Query the bug suggestions endpoint for each job
3. Aggregate results to identify known intermittents vs novel failures

API Endpoint

GET https://treeherder.mozilla.org/api/project/{repo}/jobs/{job_id}/bug_suggestions/

Returns an array of failure lines with matched Bugzilla bugs:

[
  {
    "search": "TEST-UNEXPECTED-FAIL | test.js | Test timed out",
    "path_end": "path/to/test.js",
    "bugs": {
      "open_recent": [
        {"id": 1940606, "status": "REOPENED", "summary": "Intermittent test.js | ...", "keywords": ["intermittent-failure"]}
      ],
      "all_others": [...]
    },
    "failure_new_in_rev": false
  }
]

Example Script

import urllib.request, json

headers = {'User-Agent': 'Mozilla/5.0'}
repo = "try"
job_id = 553921053

url = f"https://treeherder.mozilla.org/api/project/{repo}/jobs/{job_id}/bug_suggestions/"
req = urllib.request.Request(url, headers=headers)
with urllib.request.urlopen(req) as resp:
    suggestions = json.loads(resp.read())

for s in suggestions:
    test = s.get('path_end', '')
    open_bugs = s['bugs'].get('open_recent', [])
    all_other = s['bugs'].get('all_others', [])
    if open_bugs or all_other:
        print(f"{test}: known bugs")
        for b in open_bugs + all_other:
            print(f"  Bug {b['id']} [{b['status']}] {b['summary'][:100]}")
    else:
        print(f"{test}: NO known bugs")

Bulk Query Pattern

When checking many failing jobs (e.g., validating a new worker image), combine with treeherder-cli --json output:

# 1. Get failing jobs as JSON
treeherder-cli REVISION --repo try --platform "25h2" --json > failures.json

# 2. Extract job IDs and query bug_suggestions for each
python3 bulk_bug_check.py failures.json

Deduplicate by job_type_name to avoid querying the same test suite multiple times across retries. A User-Agent header is required for all API requests.

Key Fields

Prerequisites

• treeherder-cli: cargo install --git https://github.com/padenot/treeherder-cli
• lumberjackth: No install needed, uses uvx for zero-install execution

No authentication required for either tool.

References

• references/cli-reference.md - Complete CLI reference for both tools
• references/sheriff-workflows.md - Sheriff workflow examples
• references/api-reference.md - REST API documentation
• references/similar-jobs-comparison.md - Compare failed jobs using Treeherder's similar_jobs API

External Documentation

• Treeherder: https://treeherder.mozilla.org/
• treeherder-cli: https://github.com/padenot/treeherder-cli
• lumberjackth: https://pypi.org/project/lumberjackth/