Mid Data Engineer (Barcelona hybrid)

Barcelona·Posted 9mo ago

pythonawsspark

We are: Wizeline, a global AI-native technology solutions provider, develops cutting-edge, AI-powered digital products and platforms. We partner with clients to leverage data and AI, accelerating market entry and driving business transformation. As a global community of innovators, we foster a culture of growth, collaboration, and impact. With the right people and the right ideas, there’s no limit to what we can achieve Are you a fit? Sounds awesome, right? Now, let’s make sure you’re a good fit for the role: Responsibilities: Existing platform (Databricks) <ul> <li>Keep production pipelines running: ingestion, transformation, and delivery to downstream consumers.</li> <li>Diagnose and resolve pipeline failures and data quality issues, often without documentation to fall back on.</li> <li>Reverse-engineer and document existing transformation logic and business rules — this is the input the migration depends on.</li> <li>Migrate legacy tables from Hive Metastore to Unity Catalog.</li> <li>Maintain Iceberg-enabled table sharing between Databricks and Snowflake.</li> </ul> New development (Snowflake, dbt, Airflow) <ul> <li>Build and test dbt models, including incremental materializations and data tests.</li> <li>Develop and maintain Airflow DAGs for orchestration.</li> <li>Validate that migrated pipelines produce output equivalent to the Databricks versions.</li> <li>Contribute to Snowflake modeling, performance, and cost decisions.</li> </ul> Across both <ul> <li>Work directly with client stakeholders on technical topics, alongside the team lead.</li> </ul> Technical Requirements Databricks <ul> <li>PySpark and SQL — able to read, debug, and modify existing pipelines. Deep Spark tuning is not required.</li> <li>Delta Lake: MERGE/upsert patterns, table properties, OPTIMIZE, partitioning.</li> <li>Databricks Workflows, cluster configuration, job troubleshooting.</li> <li>Unity Catalog: catalogs, schemas, grants, lineage, and the metastore model.</li> </ul> Snowflake <ul> <li>Warehouses, roles and grants, and the general operating model.</li> <li>Query performance and an awareness of how compute cost behaves.</li> </ul> Dbt <ul> <li>Models, sources, tests, and incremental materializations.</li> <li>Project structure and how dbt fits into a deployment workflow.</li> </ul> Airflow <ul> <li>Writing and maintaining DAGs, operators, scheduling, and dependency management.</li> <li>Understanding retries, backfills, and idempotent task design.</li> </ul> Fundamentals <ul> <li>3+ years operating production data pipelines.</li> <li>Strong SQL — window functions, complex joins, reading transformation logic written by someone else.</li> <li>Python for scripting, automation, and API integration.</li> <li>Incremental loading patterns, idempotency, late-arriving data, reprocessing.</li> <li>AWS: S3, IAM basics. Basic working knowledge of Redshift and its role in the wider architecture.</li> </ul> Ways of working <ul> <li>Fluent English — client-facing role with stakeholders based abroad.</li> <li>Self-directed. Able to make progress on an unfamiliar codebase without a structured onboarding path, and comfortable asking good questions when context is missing.</li> <li>Clear communicator: can explain a production incident to a non-technical stakeholder and give a realistic ETA.</li> </ul> Nice-to-have: <ul> <li>Experience with an actual platform migration, not only greenfield work.</li> <li>Open table formats, particularly Iceberg and cross-platform sharing.</li> <li>Clickstream or web analytics data (Adobe Analytics, Google Analytics, Segment).</li> <li>Experience taking over an undocumented system and stabilizing it.</li> <li>AI Tooling Proficiency: Leverage one or more AI tools to optimize and augment day-to-day work, including drafting, analysis, research, or process automation. Provide recommendations on effective AI use and identify opportunities to streamline workflows. </li> </ul> What we offer: <ul> <li>A High-Impact Environment</li> <li>Commitment to Professional Development</li> <li>Flexible and Collaborative Culture</li> <li>Global Opportunities</li> <li>Vibrant Community</li> <li>Total Rewards</li> </ul> *Specific benefits are determined by the employment type and location. Find out more about our culture <a href="https://www.instagram.com/wizelineglobal/">here</a>.

Apply for this role