google-bigquery（Google bigquery 开发）

Google BigQuery

Access the Google BigQuery API with managed OAuth authentication. Run SQL queries, manage datasets and tables, and analyze data at scale.

Quick Start

# Run a simple query
python <<'EOF'
import urllib.request, os, json
data = json.dumps({'query': 'SELECT 1 as test_value', 'useLegacySql': False}).encode()
req = urllib.request.Request('https://api.maton.ai/google-bigquery/bigquery/v2/projects/{projectId}/queries', data=data, method='POST')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
req.add_header('Content-Type', 'application/json')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

Base URL

https://api.maton.ai/google-bigquery/bigquery/v2/{resource-path}

Maton proxies requests to bigquery.googleapis.com and automatically injects your OAuth token.

Authentication

All requests require the Maton API key in the Authorization header:

Authorization: Bearer $MATON_API_KEY

Environment Variable: Set your API key as MATON_API_KEY:

export MATON_API_KEY="YOUR_API_KEY"

Getting Your API Key

1. Sign in or create an account at maton.ai
2. Go to maton.ai/settings
3. Copy your API key

Connection Management

Manage your Google BigQuery OAuth connections at https://api.maton.ai.

List Connections

python <<'EOF'
import urllib.request, os, json
req = urllib.request.Request('https://api.maton.ai/connections?app=google-bigquery&status=ACTIVE')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

Create Connection

python <<'EOF'
import urllib.request, os, json
data = json.dumps({'app': 'google-bigquery'}).encode()
req = urllib.request.Request('https://api.maton.ai/connections', data=data, method='POST')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
req.add_header('Content-Type', 'application/json')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

Get Connection

python <<'EOF'
import urllib.request, os, json
req = urllib.request.Request('https://api.maton.ai/connections/{connection_id}')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

Response:

{
  "connection": {
    "connection_id": "{connection_id}",
    "status": "ACTIVE",
    "creation_time": "2026-02-14T09:02:02.780520Z",
    "last_updated_time": "2026-02-14T09:02:19.977436Z",
    "url": "https://connect.maton.ai/?session_token=...",
    "app": "google-bigquery",
    "metadata": {}
  }
}

Open the returned url in a browser to complete OAuth authorization.

Delete Connection

python <<'EOF'
import urllib.request, os, json
req = urllib.request.Request('https://api.maton.ai/connections/{connection_id}', method='DELETE')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

Specifying Connection

If you have multiple Google BigQuery connections, specify which one to use with the Maton-Connection header:

python <<'EOF'
import urllib.request, os, json
req = urllib.request.Request('https://api.maton.ai/google-bigquery/bigquery/v2/projects')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
req.add_header('Maton-Connection', '{connection_id}')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

If you have multiple connections, always include this header to ensure requests go to the intended account.

Security & Permissions

• Access is scoped to datasets, tables, jobs, and SQL queries within the connected Google BigQuery account.
• All write operations require explicit user approval. Before executing any create, update, or delete call, confirm the target resource and intended effect with the user.

API Reference

Projects

List Projects

List all projects accessible to the authenticated user.

GET /google-bigquery/bigquery/v2/projects

Response:

{
  "kind": "bigquery#projectList",
  "projects": [
    {
      "id": "my-project-123",
      "numericId": "822245862053",
      "projectReference": {
        "projectId": "my-project-123"
      },
      "friendlyName": "My Project"
    }
  ],
  "totalItems": 1
}

Datasets

List Datasets

GET /google-bigquery/bigquery/v2/projects/{projectId}/datasets

Query Parameters:

• maxResults - Maximum number of results to return
• pageToken - Token for pagination
• all - Include hidden datasets if true

Get Dataset

GET /google-bigquery/bigquery/v2/projects/{projectId}/datasets/{datasetId}

Create Dataset

POST /google-bigquery/bigquery/v2/projects/{projectId}/datasets
Content-Type: application/json

{
  "datasetReference": {
    "datasetId": "my_dataset",
    "projectId": "{projectId}"
  },
  "description": "My dataset description",
  "location": "US"
}

Response:

{
  "kind": "bigquery#dataset",
  "id": "my-project:my_dataset",
  "datasetReference": {
    "datasetId": "my_dataset",
    "projectId": "my-project"
  },
  "location": "US",
  "creationTime": "1771059780773"
}

Update Dataset (PATCH)

PATCH /google-bigquery/bigquery/v2/projects/{projectId}/datasets/{datasetId}
Content-Type: application/json

{
  "description": "Updated description"
}

Delete Dataset

DELETE /google-bigquery/bigquery/v2/projects/{projectId}/datasets/{datasetId}

Query Parameters:

• deleteContents - If true, delete all tables in the dataset (default: false)

Tables

List Tables

GET /google-bigquery/bigquery/v2/projects/{projectId}/datasets/{datasetId}/tables

Query Parameters:

• maxResults - Maximum number of results to return
• pageToken - Token for pagination

Get Table

GET /google-bigquery/bigquery/v2/projects/{projectId}/datasets/{datasetId}/tables/{tableId}

Create Table

POST /google-bigquery/bigquery/v2/projects/{projectId}/datasets/{datasetId}/tables
Content-Type: application/json

{
  "tableReference": {
    "projectId": "{projectId}",
    "datasetId": "{datasetId}",
    "tableId": "my_table"
  },
  "schema": {
    "fields": [
      {"name": "id", "type": "INTEGER", "mode": "REQUIRED"},
      {"name": "name", "type": "STRING", "mode": "NULLABLE"},
      {"name": "created_at", "type": "TIMESTAMP", "mode": "NULLABLE"}
    ]
  }
}

Response:

{
  "kind": "bigquery#table",
  "id": "my-project:my_dataset.my_table",
  "tableReference": {
    "projectId": "my-project",
    "datasetId": "my_dataset",
    "tableId": "my_table"
  },
  "schema": {
    "fields": [
      {"name": "id", "type": "INTEGER", "mode": "REQUIRED"},
      {"name": "name", "type": "STRING", "mode": "NULLABLE"},
      {"name": "created_at", "type": "TIMESTAMP", "mode": "NULLABLE"}
    ]
  },
  "numRows": "0",
  "type": "TABLE"
}

Update Table (PATCH)

PATCH /google-bigquery/bigquery/v2/projects/{projectId}/datasets/{datasetId}/tables/{tableId}
Content-Type: application/json

{
  "description": "Updated table description"
}

Delete Table

DELETE /google-bigquery/bigquery/v2/projects/{projectId}/datasets/{datasetId}/tables/{tableId}

Table Data

List Table Data

Retrieve rows from a table.

GET /google-bigquery/bigquery/v2/projects/{projectId}/datasets/{datasetId}/tables/{tableId}/data

Query Parameters:

• maxResults - Maximum number of results to return
• pageToken - Token for pagination
• startIndex - Zero-based index of the starting row

Response:

{
  "kind": "bigquery#tableDataList",
  "totalRows": "100",
  "rows": [
    {
      "f": [
        {"v": "1"},
        {"v": "Alice"},
        {"v": "1.7710597807E9"}
      ]
    }
  ],
  "pageToken": "..."
}

Insert Table Data (Streaming)

Insert rows into a table using streaming insert. Note: Requires BigQuery paid tier.

POST /google-bigquery/bigquery/v2/projects/{projectId}/datasets/{datasetId}/tables/{tableId}/insertAll
Content-Type: application/json

{
  "rows": [
    {"json": {"id": 1, "name": "Alice"}},
    {"json": {"id": 2, "name": "Bob"}}
  ]
}

Jobs and Queries

Run Query (Synchronous)

Execute a SQL query and return results directly.

POST /google-bigquery/bigquery/v2/projects/{projectId}/queries
Content-Type: application/json

{
  "query": "SELECT * FROM `my_dataset.my_table` LIMIT 10",
  "useLegacySql": false,
  "maxResults": 100
}

Response:

{
  "kind": "bigquery#queryResponse",
  "schema": {
    "fields": [
      {"name": "id", "type": "INTEGER"},
      {"name": "name", "type": "STRING"}
    ]
  },
  "jobReference": {
    "projectId": "my-project",
    "jobId": "job_abc123",
    "location": "US"
  },
  "totalRows": "2",
  "rows": [
    {"f": [{"v": "1"}, {"v": "Alice"}]},
    {"f": [{"v": "2"}, {"v": "Bob"}]}
  ],
  "jobComplete": true,
  "totalBytesProcessed": "1024"
}

Query Parameters:

• useLegacySql - Use legacy SQL syntax (default: false for GoogleSQL)
• maxResults - Maximum results per page
• timeoutMs - Query timeout in milliseconds

Create Job (Asynchronous)

Submit a job for asynchronous execution.

POST /google-bigquery/bigquery/v2/projects/{projectId}/jobs
Content-Type: application/json

{
  "configuration": {
    "query": {
      "query": "SELECT * FROM `my_dataset.my_table`",
      "useLegacySql": false,
      "destinationTable": {
        "projectId": "{projectId}",
        "datasetId": "{datasetId}",
        "tableId": "results_table"
      },
      "writeDisposition": "WRITE_TRUNCATE"
    }
  }
}

List Jobs

GET /google-bigquery/bigquery/v2/projects/{projectId}/jobs

Query Parameters:

• maxResults - Maximum number of results to return
• pageToken - Token for pagination
• stateFilter - Filter by job state: done, pending, running
• projection - full or minimal

Response:

{
  "kind": "bigquery#jobList",
  "jobs": [
    {
      "id": "my-project:US.job_abc123",
      "jobReference": {
        "projectId": "my-project",
        "jobId": "job_abc123",
        "location": "US"
      },
      "state": "DONE",
      "statistics": {
        "creationTime": "1771059781456",
        "startTime": "1771059782203",
        "endTime": "1771059782324"
      }
    }
  ]
}

Get Job

GET /google-bigquery/bigquery/v2/projects/{projectId}/jobs/{jobId}

Query Parameters:

• location - Job location (e.g., "US", "EU")

Get Query Results

Retrieve results from a completed query job.

GET /google-bigquery/bigquery/v2/projects/{projectId}/queries/{jobId}

Query Parameters:

• location - Job location
• maxResults - Maximum results per page
• pageToken - Token for pagination
• startIndex - Zero-based starting row

Cancel Job

POST /google-bigquery/bigquery/v2/projects/{projectId}/jobs/{jobId}/cancel

Query Parameters:

• location - Job location

Pagination

BigQuery uses token-based pagination. List responses include a pageToken when more results exist:

GET /google-bigquery/bigquery/v2/projects/{projectId}/datasets?maxResults=10&pageToken={token}

Response:

{
  "datasets": [...],
  "nextPageToken": "eyJvZmZzZXQiOjEwfQ=="
}

Use the nextPageToken value as pageToken in subsequent requests.

Code Examples

JavaScript

// Run a query
const response = await fetch(
  'https://api.maton.ai/google-bigquery/bigquery/v2/projects/my-project/queries',
  {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${process.env.MATON_API_KEY}`,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      query: 'SELECT * FROM `my_dataset.my_table` LIMIT 10',
      useLegacySql: false
    })
  }
);
const data = await response.json();
console.log(data.rows);

Python

import os
import requests

# Run a query
response = requests.post(
    'https://api.maton.ai/google-bigquery/bigquery/v2/projects/my-project/queries',
    headers={'Authorization': f'Bearer {os.environ["MATON_API_KEY"]}'},
    json={
        'query': 'SELECT * FROM `my_dataset.my_table` LIMIT 10',
        'useLegacySql': False
    }
)
data = response.json()
for row in data.get('rows', []):
    print([field['v'] for field in row['f']])

Schema Field Types

Common BigQuery data types for table schemas:

Field Modes:

• NULLABLE - Field can be null (default)
• REQUIRED - Field cannot be null
• REPEATED - Field is an array

Notes

• Project IDs are typically in the format project-name or project-name-12345
• Dataset IDs follow naming rules: letters, numbers, underscores (max 1024 characters)
• Table IDs follow same naming rules as datasets
• Job IDs are generated by BigQuery and include location prefix
• Query results use f (fields) and v (value) structure
• Streaming inserts require BigQuery paid tier (not available in free tier)
• Use useLegacySql: false for GoogleSQL (standard SQL) syntax
• IMPORTANT: When using curl commands, use curl -g when URLs contain brackets to disable glob parsing
• IMPORTANT: When piping curl output to jq or other commands, environment variables like $MATON_API_KEY may not expand correctly in some shell environments

Error Handling

Troubleshooting: API Key Issues

1. Check that the MATON_API_KEY environment variable is set:

echo $MATON_API_KEY

1. Verify the API key is valid by listing connections:

python <<'EOF'
import urllib.request, os, json
req = urllib.request.Request('https://api.maton.ai/connections')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

Troubleshooting: Invalid App Name

1. Ensure your URL path starts with google-bigquery. For example:

• Correct: https://api.maton.ai/google-bigquery/bigquery/v2/projects
• Incorrect: https://api.maton.ai/bigquery/v2/projects

Resources

• BigQuery API Overview
• Datasets
• Tables
• Jobs
• Tabledata
• Standard SQL Reference
• Maton Community
• Maton Support