WayneRecruiter Since 2001
the smart solution for Wayne jobs

Data Engineer

Company: Valley National Bank
Location: Wayne
Posted on: May 1, 2021

Job Description:

Job Description: The Technical Big Data Engineer plays a critical role in the following areas: Ingesting, transforming, storing and analyzing data in Cloudera's CDH Big Data Platform environment to support the bank's strategic business goals; Working in a development team and with other teams across the SDLC phases to ensure Valley National Bank's specific standards, procedures and methodologies are adhered to in support of the bank's strategic business goals. Responsibilities include but are not limited to:

  • Build scalable databases for the consumption of structured and unstructured data, work with external data providers to maximize competitor data throughout Valley National Bank.
  • Import and export data between an external RDBMS and the Hadoop Cloudera cluster, including the ability to import specific subsets, change the delimiter and file format of imported data during ingest and alter the data access pattern or privileges.
  • Automate data pipelines from various systems to create advanced analytics and Machine Learning.
  • Ingest real-time and near real time (NRT) streaming data into HDFS, including the ability to distribute to multiple data sources and convert data from on ingest from one format to another.
  • Convert data from one file format to another, Evolve an Avro or Parquet schema for performance optimization.
  • Develop RESTful web services.
  • Write query to filter data or to join multiple data sets.
  • Agile project methodology and waterfall where required.Requirements: Required Skills:
    • Solid track record of data integration showing excellent execution and attention to detail.
    • Programming experience ideally in Spark and a willingness to learn new languages to meet goals and objectives. Experience in Python, Kafka, Java, C, Perl, Javascript or other programming languages is a plus.
    • Experience in SQL, MySQL and SQL Server databases. Experience in MapReduce is a plus.
    • Knowledge of data mining, machine learning, natural language processing or information retrieval. Knowledge of integration concepts as they relate to sourcing data from disparate sources.
    • Experience in a cloud environment (MS Azure preferred).
    • Experience processing large amounts of structured and unstructured data, including integrating data from multiple sources.
    • Knowledge of Data warehousing best practices for optimal performance in an MPP environment.
    • Experience with Apache Avro and Apache Parquet. Experience with Impala with Kudo (interactive SQL). Experience with BI Visualization tools like MS PowerBI. Experience with test automation and QA.
    • Knowledge of Machine Learning and Computational Statistics.
    • Ability to communicate both verbally and written to employees in the business units utilizing non-technical language.
    • Willingness to explore new alternatives or options to solve data integration issues, and utilize a combination of industry best practices, data innovations and your experience to get the job done.
    • Experience in production support and troubleshooting.
    • Ability to learn to create a data-driven culture and impactful data strategies. Required Experience:
      • B.S in computer science or a related IT degree with a minimum of 2 years of big data engineering experience.
      • Working with Big Data projects.
      • Must have experience developing and integrating solutions for Big Data Platforms, including Azure.
      • Financial Services, Retail Banking, Credit Card data or related domain / industry experience of at least 2 years.
      • Cloudera Pig, Sqoop, HIVEW, MapRedue, Scala, Spark, Trifacta, etc. Cloud (Azure/AWS) experience preferred.

Keywords: Valley National Bank, Wayne , Data Engineer, Engineering , Wayne, New Jersey

Click here to apply!

Didn't find what you're looking for? Search again!

I'm looking for
in category
within


Log In or Create An Account

Get the latest New Jersey jobs by following @recnetNJ on Twitter!

Wayne RSS job feeds