Senior Data Engineer
Company: Incedo Inc.
Posted on: November 19, 2021
Sr. Data Engineer (Python and Spark, ETL, Hadoop, Big Data,
AWS)Location: 100 % Remote Skills.
- Python, Spark
- multiple cloud services - Aws Glue, Redshift, Aws RDS, S3,
CloudWatch & others.
- pyspark transformations, performance tuning.
- Experienced in airflow, cron jobs, exception handling Will be
responsible, for design, development, integration, and maintainance
of Modern Data Management systems primarily focused on Shared Disk
versus Shared Nothing use cases experienced with Snowflake, AWS
Redshift like similar MPP Databases. Build and design Moder Digital
Data Platforms in accordance with DataOps best practices and
architectural guidance, using tools like Markit, Business objects,
Informatica, MS SQL, PL/SQL, etc. Enhance/Maintain/support existing
applications. Must be hands with Snowlflake - designing Snowflake
architecture, virtual warehouse, sizing, and with a very strong
ability to communicate with stakeholders.Ability to explain the
benfits and differences between traditional datawarehouse versus
modern datawarehouses like Snowflake - shared nothing vs shared
disk databases.Must Skill:- Hands-on experience with ETL tools,
Python and Spark.- Partner with business leadership to identify
problems, and opportunities for technology innovation with a focus
on DataWarehose implementations.- Help establish a clear,
consistent technology vision through collaboration, influence, and
enablement.- Research, recommend, design and develop systems and
application architecture.- Must be an expert in SQL and one of the
ETL tools such as Informatica- Extensive experience in SQL query
tuning - Creating robust and extensible data pipelines for
production systems - Creating secure, performant, and well-modeled
data stores- Must be fluent in any one of the scripting languages
such as Python- Experience working in an onsite client technical
consulting environment preferred.- Source code version control
management using tools like Git/GitHub- Experiences working within
Agile Frameworks, such as Scrum or Kanban- Excellent communication
skills to be able to interact directly with non-technical client
stakeholders and act in a business-to-technical translation
role.Preferred Skills:- Knowledge and use of DataOps practices
preferred- Experience using NoSQL databases (e.g., AWS DynamoDB,
MongoDB, Neo4j)- Experience with metadata management, data
cataloging, data lineage, and taxonomy.
- - Experience in any popular metadata tools such as Data World,
Keywords: Incedo Inc., Trenton , Senior Data Engineer, Engineering , Dayton, New Jersey
Didn't find what you're looking for? Search again!