Data Expert
data analysis initial exploration
When reviewing or writing code, apply these guidelines:
- Begin analysis with data exploration and summary statistics.
- Implement data quality checks at the beginning of analysis.
- Handle missing data appropriately (imputation, removal, or flagging).
data fetching rules for server components
When reviewing or writing code, apply these guidelines:
- For data fetching in server components (in.tsx files): tsx async function getData() {const res = await fetch('https://api.example.com/data', {next: {revalidate: 3600}}) if (!res.ok) throw new Error('Failed to fetch data') return res.json()} export default async function Page() {const data = await getData() // Render component using data}
data pipeline management with dvc
When reviewing or writing code, apply these guidelines:
- Data Pipeline Management: Employ scripts or tools like
dvcto manage data preprocessing and ensure reproducibility.
data synchronization rules
When reviewing or writing code, apply these guidelines:
- Implement Data Synchronization:
- Create an efficient system for keeping the region grid data synchronized between the JavaScript UI and the WASM simulation. This might involve: a. Implementing periodic updates at set intervals. b. Creating an event-driven synchronization system that updates when changes occur. c. Optimizing large data transfers to maintain smooth performance, possibly using typed arrays or other efficient data structures. d. Implementing a queuing system for updates to prevent overwhelming the simulation with rapid changes.
data tracking and charts rule
When reviewing or writing code, apply these guidelines:
- There should be a chart page that tracks just about everything that can be tracked in the game.
data validation with pydantic
When reviewing or writing code, apply these guidelines:
- Data Validation: Use Pydantic models for rigorous
Consolidated Skills
This expert skill consolidates 1 individual skills:
- data-expert
Iron Laws
- ALWAYS validate all external data at system boundaries using a schema validator (Zod, Pydantic, Joi) — never trust API responses, user input, or file contents without validation.
- NEVER load entire large datasets into memory — always stream, paginate, or batch-process data beyond a few thousand records to prevent memory spikes and timeouts.
- ALWAYS sanitize data before using it in downstream operations — HTML, SQL, and shell-injected content must be stripped or escaped before processing or storage.
- NEVER use string manipulation (regex, split, replace) as a primary parser for structured formats — use purpose-built parsers (JSON.parse, csv-parse, xml2js) for reliable type-safe results.
- ALWAYS make data transformation functions pure and idempotent — a function that mutates external state or produces different results for the same input cannot be safely tested or reused.
Anti-Patterns
| Anti-Pattern | Why It Fails | Correct Approach |
|---|---|---|
| Trusting API responses without validation | API schemas change silently; unvalidated data causes downstream type errors | Validate all responses with Zod/Pydantic schemas at the API boundary |
fs.readFileSync on large CSV/JSON files | Loads entire file into memory; crashes on files > available RAM | Use streaming parsers (csv-parse/stream, JSONStream) with backpressure |
| Regex for parsing HTML or XML | HTML/XML structure is not regular; regex breaks on nested tags and attributes | Use proper DOM/XML parsers (cheerio, xml2js, DOMParser) |
| Mutating input objects in transformations | Caller still holds a reference to the mutated object; causes ghost bugs | Return new objects ({...input, newField}) instead of mutating |
| Logging full request/response bodies with PII | PII ends up in log aggregators readable by non-authorized users | Redact PII fields before logging; log schemas and IDs only |
Memory Protocol (MANDATORY)
Before starting:
cat .claude/context/memory/learnings.mdAfter completing: Record any new patterns or exceptions discovered.
ASSUME INTERRUPTION: Your context may reset. If it's not in memory, it didn't happen.