At a glance
- Experience
- 2+ years experience
- Location
- Malta
About the role
- This role will contribute in designing and developing our platform to fulfil the business needs and to also improve our systems.
- Within this capacity you will have the chance to design new data pipelines, maintain platforms hosted on data streams for batch and real-time loading and real-time visualisations.
Responsibilities
- Maintaining and evolving our existing data platform
- Building processes to ingest data from Kafka, APIs and Databases using AWS MSK Connect
- Development and Maintenance of real-time data processing applications using frameworks like Spark Structured Streaming and Kafka Streams
- Applying transformations to data streams
- Being involved in data modelling following standards e.g. Inmon, Kimball, Data Vault
- Ensuring data quality by verifying data consistency and accuracy
- Keeping up to date on research and development of new technologies and techniques to enhance our data platform
- Have an investigative mindset to be able to troubleshoot – thinking outside the box when it comes to troubleshooting problems and incident management
- Full ownership on projects and tasks assigned together with being able to work within a Team
- Able to document well processes and perform knowledge sharing sessions
Requirements
- Experience with modern cloud database technologies such as Snowflake
- Experience with orchestrating data pipelines using Airflow
- Experience with AWS Glue
- Experience with Apache Iceberg
- Experience with SQL, and Data Integration Tools
- Experience with any programming language like Python or Scala
- Knowledge on AWS Services like S3/Lambda/API Gateways/DMS/RDS
- Development experience in both Microsoft, and Linux/Cloud environments
- Have strong analytical and problem-solving skills
Preferred qualifications
- Familiar with Data Warehousing concepts and data modelling techniques such as Inmon, Kimball, Data Vault
- Familiar with AWS Services such as MSK and EMR
- Familiarity with containerization and orchestration technologies such as Docker and Kubernetes
- Familiarity with Time-series or Analytics Databases such as Elasticsearch
- Familiar with data streaming using technologies such as Kafka
- Familiar with software versioning tools like GIT
- Familiar with infrastructure scripting like Terraform
- Familiar with Data Monitoring and Visualisation tools such as Prometheus and Grafana, Cloudwatch
Skills
- Snowflake
- Airflow
- AWS Glue
- Apache Iceberg
- SQL
- Python
- Scala
- Kafka
- Spark
- Docker
- Kubernetes
- Terraform
Show all 13 skills
- Git
