(W2 - ONLY - No Sponsorship)
Job Description
We are seeking a Data Engineer with strong experience in PySpark and Databricks to design, build, and optimize scalable data pipelines and analytics solutions. The role involves working with large-scale datasets, cloud-based platforms, and modern data engineering practices to support advanced analytics, reporting, and data-driven decision-making.
Responsibilities:
Build and optimize batch and streaming data pipelines
Implement Delta Lake solutions (Delta tables, time travel, ACID transactions)
Collaborate with data scientists, analysts, and architects to deliver analytics-ready datasets
Optimize Spark jobs for performance, scalability, and cost - Integrate data from multiple sources (RDBMS, APIs, files, cloud storage
Implement data quality checks, validation, and monitoring
Manage Databricks notebooks, jobs, clusters, and workflows
Follow data governance, security, and compliance standards
Participate in code reviews and contribute to best practices
Qualifications
Technical Skills
Skill Category
Years of Experience you have
PySpark & Distributed Data Processing
Databricks Platform & Spark Optimization
Delta Lake & Medallion Architecture
AWS Data Ecosystem & Integration
Data Quality, Governance & SQL
Brandon Consulting Associates, Inc. is an EQUAL OPPORTUNITY EMPLOYER and has been in business for 29years.
...invention and progressive thinking. As an Intern, you will engage with an experienced... ...to provide full life-cycle support to AWS Data Centers from design inception through site... ...and maintenance. You will be the go to engineering resource for your region when technical...
...Requirements ~ Active Washington medical license (licensure reimbursement options available) Board certification in Podiatry Active clinical practice is required (recently retired is acceptable) Review medical records and conduct physical exams...
...Men's and Women's Cross-Country / Long-Distance Track Head Coach, Part-Time Faculty Salary: $58.05 - $78.08 Hourly Job Type: Part-Time Department: Academic Services Closing: Location: Joshua Tree, CA Job Number: 2025-12-17-PTHCCC Position...
Visitor Center and Garden Shop Manager Location Santa Barbara, CA : Title: Visitor Center and Garden Shop Manager Classification: Full-Time 40-hour work week nonexempt Department: Visitor Services Reports to: Director of Visitor Services and Education Programs...
...Network Engineer Japanese Bilingual Full-time | Entry-Level Welcome! | Japanese / English Bilingual -------------------------------- About the Role A Japanese IT company providing a wide range of technology servicesincluding system development, software...