Senior Big Data Engineer

Company Description

Deegit Inc.

Job Description

Job description:

Responsibilities

· Participate in the engineering and administration of big data systems.

· Apache Storm/Java development for both data transformation and augmentation.

· Employ best practices to ensure high availability and scalability.

· Install, configure, and maintain big data technologies and systems.

· Maintain documentation and troubleshooting playbooks.

· Respond to and resolve access and performance issues.

· Ability to work non-peak hours when needed.

· Assist in capacity planning.

· Proactively monitor performance.

· Collaborate with engineering teams to optimize data collection.

· Maintain change control and testing processes for all modifications and deployments.

· Conduct research and make recommendations on big data products, services, and standards.

Preferred Qualifications

· Experience with operational analytics and log management using ELK or Splunk.

· Understanding and experience with SQL and MPP databases such as Greenplum.

· Experience with SaltStack or comparable deployment automation tools.

· Working knowledge of Zenoss, NewRelic, PagerDuty or other monitoring technologies.

· Experience with Apache Drill, Apache Hbase, Hive, Phoenix, or other Hadoop with SQL technologies.

· Experience working with batch-processing and tools in Hadoop technology stack (MapReduce, Yarm, Pig, Hive, HDFS)

Qualifications

Basic Requirements

· BS in Computer Science, Information Systems, or related technical degree.

· 5 or more years’ experience with data transformation pipelines preferably Storm

· 5 or more years’ experience in messaging frameworks preferably Kafka.

· 5 or more years’ experience with Java programming.

Additional Information

All your information will be kept confidential according to EEO guidelines.