Data Engineer - 2522


A superb opportunity for a Data Engineer to enhance and support a federal government agency’s financial Intelligence unit data processing environment and external facing web services.

As a Data Engineer, you will

  • be working in an agile team of up to 10 people, including product owner, delivery manager, engineers and automation testers.
  • do enhancements and new data processing pipelines enhancements and new APIs, automation testing (a big focus area for the team), upgrades to underlying software components (including software fixes for breaking changes).
  • work with the product team, which has a broad mandate in relation to standardising and enhancing the agency’s core data assets. The team is responsible for the agency’s client facing APIs.


Your main skills and experience

  • Degree in Computer Science or equivalent.
  • Excellent development skills in Java and Scala (7+ years of experience)
  • In building and documenting client facing interfaces (i.e. APIs) using the springboot framework.
  • Using parallel processing technologies including Apache Spark, Kafka and the Elastic stack (Kibana, Logstash, Elasticsearch).
  • Be able to demonstrate experience in choosing & writing efficient processing algorithms.
  • Ability to write complex and optimized queries against both SQL and NoSQL data stores.
  • Transparent, collaborative when making decisions.

Nice if you have the following skills and experience

  • Federal government projects.
  • In containerisation tools and processes.
  • An understanding of Continuous Delivery techniques and tools (including Jenkins).
  • Ability to convert Behaviour-Driven Development (BDD) feature files into automated tests.

Other Requirements

  • Must be an Australian Citizen.
  • Able to obtain and maintain a Negative Vetting 1 Security Clearance.
  • Can start on or before 01/07/2019 .
  • Amenable for a contract engagement for a 12 month period and possible 12 month extension.
  • Work full time (Mon- Fri) in Chatswood NSW office.

Got questions? Email us at

Interested? APPLY NOW!


agile teamApache SparkAPIBDDBehavioural-Driven Developmentclient facing interfacescontinuous delivery techniqueselastic stackelasticsearchJavajenkinsKaftakibanalogstashMicrosoft SQLnegative vetting 1 clearanceNoSQLprocessing algorithmsScalasecurity clearance