Data Engineer

WFH, Chennai, Hyderabad

Job Code

22DE1

Experience

Minimum 3 Years

Requirements

No of Positions: 05



  • Design, develop, test, deploy, maintain and improve software needed for various projects

  • Use required technologies & web services to create, maintain, and upgrade code for secure, scalable, reliable, high performing applications and platforms in cloud and on-premise server

  • Maintain and improve data models

  • Implement test infrastructure to ensure software quality

  • Develop test automation

  • Work with cross-functional teams to ensure software quality

  • Manage project priorities, deadlines and deliverables

  • Versatile and be enthusiastic to take on new problems across the full-stack

  • Maintain semantic version control of source, software’s & resources

  • Enable data management: governance, regulatory compliance & privacy

  • Enable code required for SIEM & UEBA

  • Enable code required for ISO27001, PCI-DSS certifications of IT systems


Technical Skills:


  • Java, Spring Boot, Hibernate

  • Python, Pandas, NumPy, SciPy

  • HDFS, Apache NiFI, Airflow, Map Reduce, Sqoop, Flume, Pig, Hive, HBase, Oozie, Impala, Zookeeper, Storm, Spark, and Kafka

  • HBase, Cassandra, MongoDB, PostgreSQL

  • Data lakes & warehouse, SQL, HiveQL, GraphQL, shell scripting

  • HTML, DHTML, XML, AJAX, WSDL, SOAP, Apache Tomcat, HTML, CSS, Java Script, jQuery, Bootstrap, XML, JSON, XSL

  • Eclipse, JDBC, Toad, Maven, Ant, Jenkins, Code Deploy, GIT, Puppet, Ansible, Dockers

  • EC2, EKS, ECR, EMR, S3, Redshift, DynamoDB, Lambda, RDS, SNS, SQS

  • Linux, Unix, Windows 8, Windows 7, Windows Server 2008/2003


Soft Skills:


  • Creative and research oriented

  • Outcome driven

  • Resilient and ability to learn evolving technologies required for this job description

  • Problem solving skills

  • Business analytical thinking

  • Working proficiency in verbal and written English


Qualifications:


B.E. in C.S. or a related field and minimum 3 years experience in the following:


  • Experience working 2 or more of the following: web or mobile application development, distributed and parallel systems, Micro Services Architecture, Event driven Architecture, RESTful APIs using Swagger, In-memory computation, Parallel processing (Task & Data based approach)

  • Experienced in creating event-processing data pipelines using Kafka and Spark

  • Experienced in Real-Time Streaming applications using Kafka, Flume, Storm, and Spark Streaming

  • Data ingestion, conversion, transformation, storage using Airflow, Sqoop, NiFi, Pig, Hive

  • Good expertise in working with the Linux environment and shell Scripting

  • Experienced on Jenkins for continuous integration and deployments with different plugins like GitHub, Maven and Kubernetes

  • Hands-on experience in version control tool Git for merging and branching

  • Experience in building web services using RESTful services in Java

  • Experience in configuration, deployment, and management of enterprise applications on Application and Web Servers

  • Experience in performing Unit and Performance testing using Junit, JMeter and TestNG

  • Experience in Entity-Relationship modeling for OLTP and dimension modeling for OLAP

  • Experience in working with CSV, JSON, XML, ORC, AVRO, and Parquet file formats