Spark Developer
Niskayuna, NY
Niskayuna, NY, 12309, US
2021-06-02 12:06:32
Information Technology
Job Description
The Naval Nuclear Laboratory is seeking a Spark Developer to join a growing team dedicated to collecting and transforming data from the United States Nuclear Navy for analytics throughout the Laboratory.
As a Spark Developer, you will be part of a team responsible for designing, developing, optimizing, and maintaining the Extract, Transform, and Load (ETL) process for incoming shipboard data used in naval nuclear propulsion systems in a Hadoop environment.
You will evaluate new and emerging technologies to deliver scalable solutions to transforming shipboard data in an enterprise-wide data warehouse from which operational metrics and support analytics can be easily and reliably generated.
Preferred skills include:
- Knowledge of distributed technologies, such as Hadoop, Hive, Spark.
- Experience in Linux shell scripting and software development using languages such as SQL, Python, Scala.
- Experience in optimization and performance tuning of ETL workloads and understanding of database internals.