Apache Spark Developer

About the role

BairesDev® is seeking an experienced Apache Spark Developer to design, build, and optimise distributed data processing systems and real‑time analytics solutions. You will work on core Spark components and related ecosystem projects such as Kafka, Hive and Iceberg, contributing to performance improvements and scalable architecture.

Key responsibilities

• Contribute as a core contributor to Apache Spark and related open‑source projects.
• Develop and optimise batch and real‑time analytical workloads on distributed processing engines.
• Implement streaming analytics using Spark Structured Streaming and integrate with Apache Kafka.
• Enhance Spark SQL’s catalyst optimiser and distributed query execution.
• Collaborate with engineering teams for system integration and deployment.

Required profile

• 10+ years of software development experience.
• Strong expertise in distributed computing and large‑scale data processing.
• Deep understanding of Apache Spark architecture (RDDs, DataFrames, streaming).
• Proficiency in Scala, Java and Python for Spark development.
• Experience in performance tuning of distributed systems at scale.
• Demonstrated contributions to Apache open‑source projects (Spark, Kafka, Hive, etc.).
• Advanced level of English.

Required skills

• Scala
• Java
• Python
• Apache Spark
• Apache Kafka
• Apache Hive
• Apache Iceberg
• Spark Structured Streaming
• Spark SQL

What we offer

• Competitive compensation in USD or local currency.
• Paid parental leave, vacation and national holidays.
• Innovative, multicultural work environment.
• Mentorship, promotions, skill development and diverse growth opportunities.