Senior Databricks Engineer Job at American IT Systems, Santa Clara, CA

VnM1NHVkamV6bS9PNHYxb2g4ZG9rVVdlR0E9PQ==
  • American IT Systems
  • Santa Clara, CA

Job Description

Job Title: Senior Databricks Engineer

Location: Santa Clara. CA (5 Days Onsite)

Job Summary:

We are seeking an experienced Senior Databricks Engineer with strong expertise in Databricks, PySpark, and SQL to design, develop, and optimize large-scale data processing solutions. The ideal candidate will have hands-on experience building data pipelines, implementing ETL/ELT processes, and working with cloud-based data platforms to support analytics and business intelligence initiatives.

Key Responsibilities:

  • Design, develop, and maintain scalable data pipelines using Databricks, PySpark, Python, and SQL .
  • Build and optimize ETL/ELT frameworks for processing large volumes of structured and unstructured data.
  • Develop data ingestion solutions from multiple source systems into cloud data lakes and warehouses.
  • Optimize Spark jobs for performance, scalability, and cost efficiency.
  • Collaborate with data architects, analysts, and business stakeholders to understand data requirements.
  • Implement data quality, validation, and monitoring frameworks.
  • Perform data modeling and transformation for reporting and analytics use cases.
  • Troubleshoot production issues and provide timely resolutions.
  • Follow best practices for coding, testing, version control, and deployment.

Required Skills:

  • 8+ years of experience in Data Engineering.
  • Strong hands-on experience with Databricks.
  • Expertise in PySpark, Python, and SQL.
  • Experience with Spark optimization techniques and performance tuning.
  • Strong understanding of data warehousing concepts and dimensional modeling.
  • Experience building ETL/ELT pipelines.
  • Knowledge of Delta Lake, Spark SQL, and Databricks workflows.
  • Experience with Git, CI/CD, and Agile methodologies.
  • Strong analytical and problem-solving skills.

Preferred Skills:

  • Experience with cloud platforms such as Azure, AWS, or GCP .
  • Knowledge of Azure Data Factory (ADF), Airflow , or other orchestration tools.
  • Experience with streaming technologies such as Kafka or Structured Streaming .
  • Exposure to Power BI, Tableau, or other analytics tools.
  • Databricks or cloud certifications are a plus.

Qualifications:

  • Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
  • Excellent communication and stakeholder management skills.

Job Tags

Similar Jobs

Sage Haus

House Manager Job at Sage Haus

 ...adaptability is an important part of this role. About Our Family We are a busy entrepreneurial family seeking an experienced House Manager to support multiple closely connected residences. While most of your time will be spent at one primary home, you'll also provide... 

Chick-fil-A - North Point Village

Kitchen Team Member (Day Side) Job at Chick-fil-A - North Point Village

 ...Chick-fil-A, youll have the opportunity to work closely with a highly motivated and...  ...get your unique personality. Our Day Side Kitchen Team Member / BOH Crew Members work...  ...Crew Member is generally offered as full time or part time restaurant job. Prior experience... 

Olivier Cheng Catering and Events

Porter Job at Olivier Cheng Catering and Events

 ...garbage and recycling, and ensuring floors are swept and mopped. Polishing and maintaining silverware, copperware, wooden floors, and granite countertops, ensuring all items and surfaces remain in pristine condition. Reporting any maintenance or repair needs promptly to... 

Labcorp

Lab Assistant Job at Labcorp

 ...Labcorp seeks a Lab Assistant for second shift work at a clinical laboratory in Murfreesboro, Tennessee. The role involves receiving, preparing...  ...Six months or more phlebotomy or clinical laboratory work experience High school diploma or equivalent Ability to accurately... 

System One

Oracle Banking Product Analyst Job at System One

 ...Job Title: Oracle Banking Product Analyst Type: Contract Work Model: Remote offsite Responsibilities Lead deep system analysis...  ...vision, spending accounts, life insurance, voluntary plans, as well as participation in a 401(k) plan. System One is an Equal...