(W2 - ONLY - No Sponsorship)
Job Description
We are seeking a Data Engineer with strong experience in PySpark and Databricks to design, build, and optimize scalable data pipelines and analytics solutions. The role involves working with large-scale datasets, cloud-based platforms, and modern data engineering practices to support advanced analytics, reporting, and data-driven decision-making.
Responsibilities:
Build and optimize batch and streaming data pipelines
Implement Delta Lake solutions (Delta tables, time travel, ACID transactions)
Collaborate with data scientists, analysts, and architects to deliver analytics-ready datasets
Optimize Spark jobs for performance, scalability, and cost - Integrate data from multiple sources (RDBMS, APIs, files, cloud storage
Implement data quality checks, validation, and monitoring
Manage Databricks notebooks, jobs, clusters, and workflows
Follow data governance, security, and compliance standards
Participate in code reviews and contribute to best practices
Qualifications
Technical Skills
Skill Category
Years of Experience you have
PySpark & Distributed Data Processing
Databricks Platform & Spark Optimization
Delta Lake & Medallion Architecture
AWS Data Ecosystem & Integration
Data Quality, Governance & SQL
Brandon Consulting Associates, Inc. is an EQUAL OPPORTUNITY EMPLOYER and has been in business for 29years.
...NANNY / FAMILY ASSISTANT | WESTPORT, CT Full-Time | Live-Out | $35/hr Overnight Coverage: $... ..., Health Insurance or Stipend, Holidays Off A warm, professional family in Westport... .../week) Flexible schedule; occasional weekends or overnights Domestic and international...
...coding experience specializing in Neurosurgery or OB/GYN . Certification: Active coding certification required (CPC, CCS, RHIT, RHIA, or equivalent). OB/GYN applicants: COBGC certification through AAPC preferred. Technical Skills: Proficiency with...
...Network Engineer Japanese Bilingual Full-time | Entry-Level Welcome! | Japanese / English Bilingual -------------------------------- About the Role A Japanese IT company providing a wide range of technology servicesincluding system development, software...
Visitor Center and Garden Shop Manager Location Santa Barbara, CA : Title: Visitor Center and Garden Shop Manager Classification: Full-Time 40-hour work week nonexempt Department: Visitor Services Reports to: Director of Visitor Services and Education Programs...
...prioritizes your development. Whether youre working in our four global Home Offices, Distribution Centers or Retail StoresTJ Maxx, Marshalls, Homegoods, Homesense, Sierra, Winners, and TK Maxx, youll find abundant opportunities to learn, thrive, and make an impact. Come...