Data Integration Development
- Design and implement data integration pipelines connecting multiple data sources
- Build data transformation logic and mapping between different data formats
- Create and maintain database schemas, tables, and data models
- Optimize data pipeline performance and troubleshoot integration issues
- Develop ETL/ELT processes for real-time and batch data processing
Database Operations & Management
- Manage database operations across multiple platforms (PostgreSQL, MongoDB, ClickHouse, DuckDB, Snowflake
- Perform database administration tasks including performance tuning and optimization
- Design efficient database structures and indexing strategies
- Monitor database health, connectivity, and performance metrics
- Handle database migrations and schema evolution
Platform Operations & Automation
- Maintain Kafka topics and streaming data flows (training provided)
- Automate deployment and configuration processes using Terraform
- Monitor integration pipeline health and resolve operational issues
- Implement SRE practices for reliable data platform operations
- Document integration processes and troubleshooting procedures
Required Skills
- Must have minimum 6+ years of relevant experience
- Strong experience in stream processing using Apache Flink and messaging systems like Apache Kafka
- Hands-on experience in building real-time and batch ETL/ELT pipelines and data integration workflows
- Proficiency in Python / Java / Scala with strong data transformation and processing skills
- Experience with databases such as PostgreSQL, MongoDB, ClickHouse, DuckDB, and Snowflake
- Strong understanding of data modeling, indexing, and performance tuning
- Experience with pipeline monitoring, troubleshooting, and optimization
- Knowledge of Terraform and CI/CD for automation and deployment
- Understanding of SRE practices and platform reliability
- Good problem-solving, debugging, and documentation skills
