Data Engineer (AWS)
BSC is a technology and analytics driven consulting company empowering our clients by solving real-world business challenges that deliver tangible value.
Our core capabilities span the disciplines of Data Science, Data Engineering and Software Engineering.
Job Overview
- You will work within a team solving complex problems to deliver real business value across a wide range of industries such as mining, telco’s, retail and financial services.
- You will help develop best practices and drive improvements in data engineering across the business.
Responsibilities, Activities and Key Delivarables
- Analyse and organise raw data
- Design data engineering solutions to meet business requirements
- Build scalable data pipelines that clean, transform and aggregate data from different sources using appropriate tools and technologies
- Collaborate with data scientists to prepare data sets for analytical modelling
- Identify ways to enhance data quality and reliability
- Manage technical delivery of projects
- Mentor junior data engineers
Job Specific Qualifications and Experience
Qualifications
- Graduate or Postgraduate degree in Computer Science, Information Technology, Statistics, Engineering or related technical field.
Requirements
- 2-6 years of relational SQL database
- Solid understanding of Apache Spark as a distributed data processing framework
- Experience in designing, implementing and performance tuning solutions in Apache Spark
- Experience in complex data warehouse and data lake design
- Experience in building, optimising and managing data pipelines for large and complex datasets
- Experience using AWS cloud data platform technologies:
- Amazon S3
- Amazon Redshift
- AWS Glue
- Amazon Athena
- Amazon Elastic MapReduce (EMR)
- AWS Step Functions
- AWS Lambda to implement data pipelines and analytical solutions
- Experience in Python would be beneficial
- Experience with streaming technologies such as Spark streaming or Kafka would be beneficial
- A relevant data engineering certification would be beneficial
Behavioral Competencies
- Problem Solver: Can identify problems and opportunities and use critical thinking skills to develop innovative solutions in response.
- Client Focused: Seeks to understand clients and establish effective collaborative working relationships.
- Proactive: Communicates proactively with project stakeholders and takes action to achieve goals.
- Resilient: Copes effectively under pressure or opposition (time or ambiguity).
- Team Orientated: Enjoys working in a team and has a collaborative and inclusive leadership style.
- Quality Focused: Focused on delivering high quality solutions.
- Communication Skills: Strong verbal and written communication skills. Can effectively present complex technical concepts to both technical and non-technical audiences.
Assesment Process
As part of our commitment to a thorough and fair selection process, we utilise assessments to evaluate the suitability of candidates for specific roles.
These assessments, designed to understand individual job-related behaviour and characteristics in comparison to a role benchmark, ensure that all candidates are evaluated based on their potential job fit. This process compliments the review of resumes, academic transcripts, and other relevant qualifications.
The link to the assessments will be sent to candidates via email.
To ensure your application is complete, please make sure the following items are submitted:
- CV Uploaded
- Transcripts
- Degree Uploaded
- Assessments Completed
- All Questions Answered
Applications with missing information will not be considered.
We ensure that our recruitment process adheres to best practices and legal standards, respecting the privacy and rights of all applicants.