Data Type Converter
Convert data between JSON, CSV, XML, YAML, and TOML formats. Handles nested structures, arrays, and complex data with intelligent flattening options.
Quick Start
from scripts.data_converter import DataTypeConverter
# JSON to CSV
converter = DataTypeConverter()
converter.convert("data.json", "data.csv")
# YAML to JSON
converter.convert("config.yaml", "config.json")
# With options
converter.convert("data.json", "data.csv", flatten=True)
Features
- 5 Formats: JSON, CSV, XML, YAML, TOML
- Nested Data: Flatten or preserve nested structures
- Arrays: Handle array data intelligently
- Type Preservation: Maintain data types where possible
- Pretty Output: Formatted, human-readable output
- Batch Processing: Convert multiple files
API Reference
Basic Conversion
converter = DataTypeConverter()
# Auto-detect format from extension
converter.convert("input.json", "output.csv")
converter.convert("input.xml", "output.json")
converter.convert("input.yaml", "output.toml")
With Options
# Flatten nested structures for CSV
converter.convert("nested.json", "flat.csv", flatten=True)
# Pretty print output
converter.convert("data.json", "pretty.json", indent=4)
# Specify root element for XML
converter.convert("data.json", "data.xml", root="records")
Programmatic Access
# Load and convert in memory
data = converter.load("data.json")
converter.save(data, "data.yaml")
# String conversion
json_str = '{"name": "John", "age": 30}'
yaml_str = converter.convert_string(json_str, "json", "yaml")
Batch Processing
# Convert all JSON files to CSV
converter.batch_convert(
input_dir="./json_files",
output_dir="./csv_files",
output_format="csv"
)
CLI Usage
# Basic conversion
python data_converter.py --input data.json --output data.csv
# With flattening
python data_converter.py --input nested.json --output flat.csv --flatten
# Batch convert
python data_converter.py --input-dir ./json --output-dir ./csv --format csv
# Pretty print
python data_converter.py --input data.json --output pretty.json --indent 4
CLI Arguments
| Argument | Description | Default |
|---|
--input | Input file | Required |
--output | Output file | Required |
--input-dir | Input directory for batch | - |
--output-dir | Output directory | - |
--format | Output format | From extension |
--flatten | Flatten nested data | False |
--indent | Indentation spaces | 2 |
--root | XML root element | root |
Conversion Matrix
| From/To | JSON | CSV | XML | YAML | TOML |
|---|
| JSON | - | Yes | Yes | Yes | Yes |
| CSV | Yes | - | Yes | Yes | Yes |
| XML | Yes | Yes | - | Yes | Yes |
| YAML | Yes | Yes | Yes | - | Yes |
| TOML | Yes | Yes | Yes | Yes | - |
Examples
JSON to CSV (Flat)
converter = DataTypeConverter()
# Input: data.json
# [{"name": "John", "age": 30}, {"name": "Jane", "age": 25}]
converter.convert("data.json", "data.csv")
# Output: data.csv
# name,age
# John,30
# Jane,25
Nested JSON to Flat CSV
# Input: nested.json
# [{"user": {"name": "John", "email": "j@test.com"}, "orders": 5}]
converter.convert("nested.json", "flat.csv", flatten=True)
# Output: flat.csv
# user.name,user.email,orders
# John,j@test.com,5
YAML Config to JSON
# Input: config.yaml
# database:
# host: localhost
# port: 5432
# debug: true
converter.convert("config.yaml", "config.json")
# Output: config.json
# {"database": {"host": "localhost", "port": 5432}, "debug": true}
XML to JSON
# Input: data.xml
# <users>
# <user><name>John</name><age>30</age></user>
# </users>
converter.convert("data.xml", "data.json")
# Output: data.json
# {"users": {"user": {"name": "John", "age": "30"}}}
Dependencies
pyyaml>=6.0
toml>=0.10.0
xmltodict>=0.13.0
pandas>=2.0.0
Limitations
- CSV doesn't support nested data (requires flattening)
- XML attribute handling is basic
- TOML doesn't support null values
- Very deep nesting may cause issues with some formats
- Array handling varies by format
