Company: Valley National Bank
Posted on: May 1, 2021
Job Description: The Technical Big Data Engineer plays a
critical role in the following areas: Ingesting, transforming,
storing and analyzing data in Cloudera's CDH Big Data Platform
environment to support the bank's strategic business goals; Working
in a development team and with other teams across the SDLC phases
to ensure Valley National Bank's specific standards, procedures and
methodologies are adhered to in support of the bank's strategic
business goals. Responsibilities include but are not limited
- Build scalable databases for the consumption of structured and
unstructured data, work with external data providers to maximize
competitor data throughout Valley National Bank.
- Import and export data between an external RDBMS and the Hadoop
Cloudera cluster, including the ability to import specific subsets,
change the delimiter and file format of imported data during ingest
and alter the data access pattern or privileges.
- Automate data pipelines from various systems to create advanced
analytics and Machine Learning.
- Ingest real-time and near real time (NRT) streaming data into
HDFS, including the ability to distribute to multiple data sources
and convert data from on ingest from one format to another.
- Convert data from one file format to another, Evolve an Avro or
Parquet schema for performance optimization.
- Develop RESTful web services.
- Write query to filter data or to join multiple data sets.
- Agile project methodology and waterfall where
required.Requirements: Required Skills:
- Solid track record of data integration showing excellent
execution and attention to detail.
- Programming experience ideally in Spark and a willingness to
learn new languages to meet goals and objectives. Experience in
languages is a plus.
- Experience in SQL, MySQL and SQL Server databases. Experience
in MapReduce is a plus.
- Knowledge of data mining, machine learning, natural language
processing or information retrieval. Knowledge of integration
concepts as they relate to sourcing data from disparate
- Experience in a cloud environment (MS Azure preferred).
- Experience processing large amounts of structured and
unstructured data, including integrating data from multiple
- Knowledge of Data warehousing best practices for optimal
performance in an MPP environment.
- Experience with Apache Avro and Apache Parquet. Experience with
Impala with Kudo (interactive SQL). Experience with BI
Visualization tools like MS PowerBI. Experience with test
automation and QA.
- Knowledge of Machine Learning and Computational
- Ability to communicate both verbally and written to employees
in the business units utilizing non-technical language.
- Willingness to explore new alternatives or options to solve
data integration issues, and utilize a combination of industry best
practices, data innovations and your experience to get the job
- Experience in production support and troubleshooting.
- Ability to learn to create a data-driven culture and impactful
data strategies. Required Experience:
- B.S in computer science or a related IT degree with a minimum
of 2 years of big data engineering experience.
- Working with Big Data projects.
- Must have experience developing and integrating solutions for
Big Data Platforms, including Azure.
- Financial Services, Retail Banking, Credit Card data or related
domain / industry experience of at least 2 years.
- Cloudera Pig, Sqoop, HIVEW, MapRedue, Scala, Spark, Trifacta,
etc. Cloud (Azure/AWS) experience preferred.
Keywords: Valley National Bank, Wayne , Data Engineer, Engineering , Wayne, New Jersey
Didn't find what you're looking for? Search again!