Data Engineer

Role and Responsibilities

• Create and maintain optimal data pipeline architecture.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Databricks technologies.
• Work with stakeholders including the executive, product, data, and design teams to assist with data-related technical issues and support their data infrastructure needs.
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.
• Assemble large, complex data sets that meet functional and non-functional business requirements.
• Monitor and maintain reliability of big data platform and related tools.
• Collaborate with cross-functional teams to deliver the data features and products.
• Control and automate the deployment process of data API, job, and data pipeline on the platform.

Qualifications :

• 5 years of experience in building and optimizing ‘big data’ data pipelines, architectures, and data sets.
• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) and working familiarity with various databases.
• Strong analytic skills related to working with unstructured datasets.
• Experience supporting and working with cross-functional teams in a dynamic environment.
• Experience with big data tools: Hadoop, Hive, Spark, Presto.
• Experience with cloud services: AWS, Microsoft Azure is a plus.
• Experience with relational SQL and NoSQL databases such as Postgres, Cassandra and MongoDB is a plus.