Data Engineer (Databricks & Python)
About Us
BySix Engineers the future and drives innovation. We are more than just a standard software engineering company, we believe that true impact comes from combining technical expertise with human values. That's why we reflect that in our approach - technology is not the end goal, but the enabler.
Why Us?
🫸 Multidisciplinary team
📒 Training on demand
💴 Attractive salary and benefits package
📈 Career progression plans
👌 And amazing get together events
What are we looking for?
We are looking for a Data Engineer with:
- Minimum 10 years of professional experience developing in Python.
- At least 5 years of hands-on experience with Databricks, including PySpark development in production environments.
- Proven experience working as a member of Scrum or Agile teams.
- Solid experience designing Python libraries, frameworks, or shared components.
- Strong knowledge of software engineering best practices, including: Object-Oriented Programming (OOP), Design patterns, Unit and integration testing, CI/CD pipelines
- Experience with code standardization and quality tools, such as linting and formatting tools (e.g., pylint, flake8, black or equivalent).
- Strong understanding of batch and streaming data processing.
- Experience with Medallion Architecture and data lifecycle best practices.
- Familiarity with Airflow, Terraform, and Azure ADLS Gen2.
- Professional working proficiency in English, both written and spoken.
- Strong communication and collaboration skills in cross-functional and multicultural teams.
- Sense of ownership and accountability for delivered solutions.
We Value
- Databricks Certified Associate or Professional certification.
- Microsoft Azure Fundamentals (AZ-900) or equivalent basic Azure certification.
- Experience contributing to shared platforms or internal frameworks used by multiple teams.
- Experience working in international or distributed environments.
What do we expect from you?
- Design, implement, and maintain a shared Python library for Databricks, supporting batch and streaming pipelines.
- Develop reusable PySpark modules, base classes, and abstractions for Bronze, Silver, and Gold layers.
- Actively participate as a Scrum team member in Sprint Planning, Daily, Refinement, Review, and Retrospective ceremonies.
- Define and enforce software engineering best practices, including coding standards, documentation, testing strategies, and versioning.
- Establish and maintain code quality standards, including linting, formatting, and static analysis.
- Collaborate with Product Owners and fellow engineers to clarify requirements and deliver incremental value.
- Maintain and improve CI/CD pipelines using GitLab and Databricks Asset Bundles (DABs).
- Ensure controlled releases, backward compatibility, and smooth adoption of the common library across teams.
- Integrate logging, monitoring, and data quality controls using Grafana and DQX.
- Work closely with DataOps to ensure stability, observability, and reliability in production environments.
Are you ready?
If you're ready to make a real impact by leveraging cutting-edge technology and fostering human-centric solutions, BySix is the place for you. Together, we'll drive innovation and create lasting business value.
Note: BySix is an equal opportunity employer. All applicants will be considered and analyzed regardless of ethnicity, religion, gender identity, sexual orientation, national origin, age, or disability status.
person_pinLocation
Leiria (Hybrid)
historyExperience
Min. 10 years
architectureMain skills
