Job Description

Role: Data Engineer

Duration: Long-term project.

Location: Redmond WA

Key Skills

• Apache Spark (PySpark or Scala) : Writing scalable distributed rule engines.
• Apache Hive : Querying partitioned datasets, schema handling.
• Amazon EMR : Cluster management, tuning jobs for large datasets (20+ TB).
• AWS S3 : Data partitioning, object lifecycle, access patterns.
• Glue Data Catalog : Managing Hive table metadata.
• Data Quality Frameworks : (Great Expectations, Deequ, or custom rules).
• CI/CD : Code deployment pipelines (Git, Jenkins, Airflow, etc

Job Tags

Similar Jobs