lead data architect

Karsun Solutions, LLC • Philadelphia, PA • 2h ago

Why Karsun?

Join Karsun Solutions to grow your career with the company transforming possible for the US Government.

At Karsun, collaboration drives our community. We're committed to building an environment where team members from diverse backgrounds can innovate, learn and grow with us. Here at Karsun, the only limit to your potential is the limit of your curiosity.

Join Team Karsun, and Find Your Next!

Summary

The Lead Data Architect will design, build, and operate enterprise data

platforms that power GenAI and AI/ML use cases. This is a highly technical, hands-on

role responsible for data platform architecture, end-to-end data engineering, ML/LLM

pipeline design, production model onboarding, and delivery of scalable Databricks-

centric solutions across cloud environments

What You'll Be Doing:

Architect and implement enterprise data platforms (batch + streaming) optimized for ML, LLMs, and GenAI workloads.

Lead design and hands on implementation of Databricks workspaces, Unity Catalog, Delta Lake design patterns, cluster policies, and performance tuning.

Build and own end to end data pipelines (ingest, transform, feature engineering, serving) using PySpark, Databricks Jobs, Spark SQL, Delta Lake, and orchestration tools.

Design and operationalize model training, fine tuning (LLM), evaluation, deployment, and monitoring pipelines (MLOps/RAG/CAG) integrating Databricks MLflow, CI/CD, and infra-as-code.

Implement vectorless and vectorization/embedding pipelines, vector store integrations, and retrieval layers for RAG (FAISS, Pinecone, Weaviate, Milvus).

Define data schemas, governance, lineage, access controls, and data product APIs; implement Unity Catalog or equivalent for centralized governance.

Drive cost/performance optimization for storage, compute (spot/preemptible),and query patterns.

Collaborate with engineers, data scientists, product owners, and security to translate business needs into production GenAI solutions.

Mentor and lead engineering teams; conduct architecture reviews, code reviews, and run technical deep dives.

Implement observability for data and ML pipelines (metrics, logging, data quality tests, alerting).

Create reproducible experiment tracking, model registry, and rollout strategies (canary, shadow testing, rollback).

Stay current on GenAI/LLM architectures and evaluate/introduce new tooling and frameworks.

Required Qualifications:

BA or BS degree in CS, Computer Engineering, Information Technology or arelated field.

8+ years hands on experience in data engineering/platform architecture; 3+ years in an architect or lead role.

Candidate must hold an active AWS Certified Machine Learning - Specialty certification or equivalent AWS certification.

Proven, hands on Databricks experience (designing workspaces, Delta Lake, performance tuning, productionizing Spark jobs).

Deep Spark + PySpark expertise and experience with Databricks Runtime.

Strong experience building ML/LLM pipelines and operationalizing models (training, fine tuning, serving).

Practical experience with vector embeddings, semantic search, and RAG architectures.

Solid Python expertise and common ML libraries (PyTorch, TensorFlow, Hugging Face transformers) and MLflow.

Cloud platform experience (AWS strongly preferred).

Experience with containerization and orchestration while leveraging open source libraries for unstructured and structured data processing, serving/inference.

Strong SQL skills; experience with distributed query/warehouse systems and parquet/AVRO/Delta formats.

CI/CD and infra-as-code experience (Terraform, GitOps, Jenkins/GitHub Actions/GitLab CI).

Data governance, security, and IAM experience; experience implementing row/column level access controls and data lineage.

Demonstrated ability to design for scalability, reliability, and cost efficiency.

Preferred Qualifications:

Prior experience with Databricks Unity Catalog, Photon, and Databricks SQL.

Experience integrating Databricks with vector databases (Pinecone, neo4j) and retrieval frameworks (LangChain, LlamaIndex).

Familiarity with AWS Bedrock or other managed LLM services.

Experience with realtime streaming (Kafka, Kinesis) and stream processing on Databricks Structured Streaming.

Certifications: Databricks Certified Professional.

Experience with data quality and profiling tools (Great Expectations, Soda).

Experience with large-scale ETL frameworks and tools (Airflow, Prefect).

Things to Know:

Commitment to Non-Discrimination

All qualified applicants will receive consideration for employment without regard to disability, status as a protected veteran or any other status protected by applicable federal, state, local, or international law.

Salary Range

The proposed salary range for this role is $160,000 to $190,000 USD. The salary range provided is a good faith estimate representative of all experience levels. Karsun considers several factors when extending an offer, including but not limited to, the role, function and associated responsibilities, a candidate's work experience, location, education/training, and key skills.

Third Party Resumes:Karsun does not accept unsolicited resumes through or from search firms or staffing agencies. All unsolicited resumes will be considered the property of Karsun and Karsun will not be obligated to pay a placement fee.

Clearance Information

This position requires the eligibility to obtain a security clearance. The Defense Industrial Security Clearance Office (DISCO), an agency of the Department of Defense, handles and adjudicates the security clearance process. More information about Security Clearances can be found on the US Department of State government website:https://www.state.gov/m/ds/clearances/c10978.htm

Location

To be considered for this role, you must reside in one of the following states: CA, CO, DC, FL, GA, IL, MD, NJ, NY, NC, OH, OK, PA, SC, TX, VA, WV.

Work Authorization

Applicants must be authorized to work in the U.S. We may consider candidates currently in H-1B status who are eligible for transfer.

Statement on AI and Hiring Process

At Karsun, we are committed to a fair and equitable hiring process. We do not use artificial intelligence to make decisions on candidates, scan resumes, or source potential hires. Applicants are reviewed by our talent acquisition team and hiring managers to ensure a thorough evaluation

Similar Jobs

Mechanical Engineering New Graduate | Broomall, PA

IMEG Corp. • Broomall, PA • 1h ago

IT & Engineering

Mechanical Engineering New Graduate | Broomall, PA 100% Employee-Owned | Big-firm resources, local-team relationships, ONE IMEG culture This posting is for Summer 2027 New Graduate opportunities. IMEG is hiring a Mechani...

1h ago

Go to Apply

Mechanical Engineer I

Belcan • Philadelphia, PA • 1h ago

IT & Engineering

Mechanical Engineer I Job Number: 367610 Category: Mechanical Engineering Description: Job Title: Mechanical Engineer I Location: Philadelphia, PA Zip Code: 19135 Start Date: Right Away Pay Rate: $42.89/hr. Contract: 4 M...

1h ago

Go to Apply

Senior Forensic Engineer - Mechanical

The Vertex Companies LLC • Chester, PA • 1h ago

IT & Engineering

Company Description The Vertex Companies, LLC (VERTEX) is a $180M global consulting firm that integrates strategic advisory, project management, and dispute resolution services for organizations facing complex challenges...

1h ago

Go to Apply

Engineer III - Electrical

Qualus • Doylestown, PA • 3h ago

IT & Engineering

Position Overview Power your future with Qualus in our Engineering department as an Engineer III - Electrical. As Electrical Engineer you will work with several departments in different geographies to perform power syste...

3h ago

Go to Apply

Electrical Engineering New Graduate | Broomall, PA

IMEG Corp. • Broomall, PA • 3h ago

IT & Engineering

Electrical Engineering New Graduate | Broomall, PA 100% Employee-Owned * Big-firm resources, local-team relationships, ONE IMEG culture This posting is for Summer 2027 New Graduate opportunities. IMEG is hiring an Electr...

3h ago

Go to Apply

Principal Software Engineer

ASRC Federal Holding Company • Moorestown, NJ • 8h ago

IT & Engineering

ASRC Federal is a leading government contractor furthering missions in space, public health and defense. As an Alaska Native owned corporation, our work helps secure an enduring future for our shareholders. Join our team...

8h ago

Go to Apply

Principal Software Engineer

ASRC Federal Holding Company • Moorestown, NJ • 8h ago

IT & Engineering

8h ago

Go to Apply

Software Engineer

Actalent • Moorestown, NJ • 8h ago

IT & Engineering

Job Title: Software Engineer Job Description This role focuses on designing, developing, troubleshooting, and analyzing software programs for computer-based systems within a complex defense environment. You will perform...

8h ago

Go to Apply

2027 Early Talent Program - Global Technology & Solutions (Intern Conversion)

TD Bank • Mount Laurel, NJ • 9h ago

IT & Engineering

On Site Mount Laurel, New Jersey Full time R_1508099 Role Type: Undergraduate Work Location : Mount Laurel, New Jersey, United States of America Hours: 40 Pay Details: $35.00 - $35.00 USD TD is committed to providing fai...

9h ago

Go to Apply

Transportation Engineer - Entry-Level

Whitman, Requardt and Associates, LLP • King Of Prussia, PA • 13h ago

IT & Engineering

Key Responsibilities Include: Highway Engineering Assist in the development of roadway alignments, profiles, and typical sections. Prepare construction plans for roadway geometrics, maintenance of traffic (MOT) and erosi...

13h ago

Go to Apply

Subscribe to get new jobs sent directly to you
(no charge, cancel anytime)