Duties:

Act as the principal data architect, designing core data structures that support end-to-end logistics operations and sustainability analytics.

Lead the development and maintenance of scalable data pipelines and ETL workflows using Apache Spark, PySpark, and SQL.

Architect and manage large-scale distributed data processing systems, leveraging technologies such as Databricks, Delta Lake, and Hadoop.

Ingest and process data from diverse sources including APIs, cloud storage, relational databases, and flat files (structured and unstructured).

Optimize data models and architecture to enable robust analytics, BI reporting, and machine learning workflows.

Collaborate closely with cross-functional teamsincluding data scientists, analysts, and application developersto ensure data quality and accessibility.

Build and maintain both batch and streaming data pipelines, ensuring high performance, scalability, and fault tolerance.

Education and Experience :

Bachelor of Science degree in Computer Science, Engineering (any branch), Information Systems/Technology, Computer Applications/Applied Computer Science, or Electrical/Electronic Engineering plus five (5) years of experience as Systems Architect/Engineer, Software Developer/Engineer, Computer Systems Analyst or related position. Foreign educational equivalent accepted.

Skills Required:

(1) Azure Databricks, Azure Data Factory, Spark Big Data, Hive (2) PostgreSQL, Oracle, Teradata (3) Apache Kafka, Azure EventHub, Azure Functions (4) Apache Spark, Delta Lake (5) Informatica Power Center, Informatica Data Quality, Unix shell script

GC Worksite:

5550 Granite Pkwy., Suite 100, Plano, TX 75024 and other unanticipated worksites throughout the United States.

INFOGAIN CORPORATION

Latest Jobs

Data Engineer