Senior Data Engineer

Where are we headed:

• We’re building the foundation of decentralized knowledge graph infrastructure for web3 and developing high quality products to showcase this.
• Our road-map is robust, and with an iterative and continuous development philosophy, we’re always building.
• We’re a global team that values diverse perspectives and top tier talent. We’re growing internationally with the support of top tier investors and advisors.

Where do you fit in?

We are seeking a seasoned Data Engineer to play a crucial role in building and optimizing our data products and pipelines. The individual will be tasked with developing, constructing, testing, and maintaining architectures such as databases and large-scale processing systems. Utilizing modern data orchestration tools, the Data Engineer will streamline the building of streaming and batch processed data feeds, and integrate data tools into AI models and analytics tools.

Responsibilities

Data Pipeline Development and Optimization

• Design, construct, install, and maintain large-scale processing systems and other infrastructure.
• Develop data set processes for data modeling, mining, and production.
• Employ a variety of languages and tools to marry systems together.
• Implement data flows that connect operational systems, data analytics, and data visualization systems.

Data Integration

• Build, develop, and maintain processes that import/extract, analyze, and visualize data, making it available to analysts and other end-users.
• Ensure systems meet business requirements and industry practices.
• Work towards the integration of data analytics and AI models.

Data Testing and Deployment

• Create reliable, automated processes for verifying data quality and integrity.
• Deploy sophisticated analytics programs, machine learning, and statistical methods.
• Utilize CI/CD methodologies to ensure smooth transitions of data products from development to production.

Data Scalability and Performance Optimization

• Optimize SQL, dags, or other data pipeline operations as necessary and ensure data solutions are scalable.
• Ensure data privacy and compliance with data governance and data management principles.
• Employ strategies for managing and processing large data sets effectively and efficiently.

Data Exploration and Visualization

• Perform exploratory data analysis and visualization for initial insights and data familiarization.
• Employ a myriad of tools and platforms for connecting disparate data sources and visualizing complex datasets.

Skills and Qualifications

Essential

• Must have web3 data related work experience
• Data Processing: Proficiency in managing large datasets, ranging from hundreds of GB to TB, with billions of entries.
• Programming: Extensive use and knowledge of SQL and Python.
• Data Orchestration: Experience with data orchestration tools like Apache Airflow or Dagster.
• Data Streaming: Proficiency in managing and manipulating streaming data - e.g., experience with Kafka.
• Cloud Platforms: Hands-on experience with cloud platforms like GCP or AWS.

Desirable

• Data Visualization: Experience with data visualization tools such as Power BI.
• CI/CD: Familiarity with CI/CD practices and tools for data.
• Machine Learning Integration: Experience integrating machine learning models into data pipelines.

Educational & Experience Requirements:

• Bachelor’s degree in Computer Science, Engineering, or related field.
• A minimum of 3 years of experience in a Data Engineering role.
• Fluent English, both written and spoken
• Significant experience collaborating within an engineering team and reviewing code
• Have exceptional technical communication skills (written and verbal)

Write to Us If You:

• Relish problem-solving and are constantly seeking out challenging coding tasks to tackle.
• Have an interest in blockchain technology and the transformative potential it holds.
• Are committed to working like an owner, going beyond the confines of a conventional 9-to-5 role or contractor mentality.
• Are comfortable assuming significant responsibility and ownership, and thrive in an environment where you’re trusted to make key decisions.