Data Engineer

Karsun Solutions

United States Remote

Full time

$98-126k (annually)

Jan 17

The Data Engineer is responsible for the maintenance, synchronization, cleaning, and migration of transactional data in a hybrid environment with both on-prem and highly modern cloud based microservices environment. The Data Engineer works with the product teams to understand, analyze, document and efficiently implement to deliver streaming as well as batch oriented data for synchronizing legacy and modern data stores ensuring data integrity.

 

The Data Engineer provides support to the Application database design to aid in eliminating data duplication and other issues and enabling selective event based and scheduled based data transfer to endpoints within the cloud and legacy environment as required. The Data Engineer drives towards programmatic pipeline generation and orchestration to enhance repeatability and rapid deployment, using out of the box thinking, AWS native capabilities and CI/CD tools while utilizing established design patterns and methods.

  • The successful candidate will be able to rapidly develop technical solutions working closely with the integrated product teams and developers with minimal direction from senior or lead resources
  • Understand data needs and be able to construct data pipelines for automating event driven bi-directional selective data replication, along with micro-batch and batch- based data pipelines
  • Standardization of data processing modules to deliver modularity and enhance reusability
  • Understand, maintain and perform operation on datasets stored in relational databases such as MySQL, SQL Server and Redshift, as well as in AWS S3
  • Utilize data processing tools and services such as StreamSets, Python, Java and shell scripting, AWS tools such as Glue, Step functions and Lambda and DB tools such as MySQL Workbench and SQL Server Management Studio
  • Create and maintain standards and best practices for data and pipeline standards
  • The candidate will be expected to support a variety of structured, semi-structured and unstructured data in streaming and batch frameworks
  • Responsible for the design, creation and support of data pipeline ETl processes across various data assets within the current scope of the system, able to monitor, troubleshoot and coordinate defect resolution related to data processing & preparation.

Required:

  • Typically requires a bachelor's degree or higher in Computer Science or related discipline
  • 4+ years of hands-on experience in data transformation, creation of complex SQL queries and functions as well as data processing, cleanup and migration (MySQL and SQL Server preferred)
  • 3+ years of hands-on experience in with Python and various Python toolkits and libraries for data processing and pipelines
  • 3+ years of Database operations and administration (MySQL and SQL Server preferred)
  • 2+ years of hands-on experience in with ETL tools such as Streamsets, AWS Glue, Pentaho etc.
  • 1+ year of hands-on experience in AWS
  • 1+ year of experience in Java and Linux scripting
  • 1+ year experience working with CI/CD tools including Git
  • Successful track record in Data migrations, Database operations and maintenance and ETL job design and development, as well as scripting and automation activities with minimal supervision. 
  • Skills in database operations, such as Export/Import, Backup/Restore etc. as well as proven experience with AWS data tools and services related to Data processing.

 

Desired:

  • Experience of working with multiple AWS tools and any AWS certification is highly desired
  • Advanced Database Administration skills in SQL Server, MySQL and Redshift is highly desired
  • Experience with big data tools such as EMR/Spark, Databricks/PySpark is an advantage
  • Experience working with database versioning tools and tools like SSRS is an advantage
  • Experience in Ansible and Jenkins scripting is an advantage

 

 

In accordance with pay transparency guidelines, the proposed salary range for this position is $98,000 to $126,000. Final salary will be determined based on various factors such as relevant skills, experience and certifications. 

 

At Karsun, we celebrate your contributions, provide you with opportunities and support your total well-being. Our comprehensive benefit offerings include healthcare, retirement plan, insurance programs, commuter program, employee assistance program, paid and unpaid leave programs, supplemental benefit offerings and training. 

ApplyReferShare Job





Karsun Solutions is an Equal Employment Opportunity (EEO) employer. It is the policy of the Company to provide equal employment opportunities to all qualified applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, protected veteran or disabled status, or genetic information.

Apply for this position Back to job

You must be logged in to to apply to this job.

Apply

Your application has been successfully submitted.

Please fix the errors below and resubmit.

Something went wrong. Please try again later or contact us.

Personal Information

Profile

View resume

Details

Karsun Solutions

The Enterprise Modernization Experts delivering innovative software, cloud and data solutions to US government customers.

{{notification.msg}}