Experience
- Strong experience with the deployment, configuration, and operationalization of Databricks environments, including workspace architecture, cluster management, CI/CD integration, security, governance, and enterprise-scale administration.
- Experienced in building and managing modern data pipelines and lakehouse architectures using Delta Lake, Delta Live Tables, Structured Streaming, Workflows, medallion architectures (Bronze/Silver/Gold), and real-time/batch ingestion frameworks.
- Deep understanding of Databricks ecosystem components including Unity Catalog, data lineage, RBAC, monitoring/observability, cost optimization, ML/AI enablement, model serving, and secure enterprise data collaboration through Clean Rooms.
- Proven experience integrating Databricks with enterprise cloud and industrial data ecosystems, including Kafka, SQL databases, APIs, IoT/OT platforms, and cloud environments such as Azure, AWS, and GCP.
- Strong understanding of scalable data engineering, governance, multi-tenant architectures, and enterprise data platform strategies supporting analytics, AI, and operational intelligence initiatives.
- Proficiency in programming languages like Python, R, or Java
- Experience with data manipulation and analysis libraries (e.g., Pandas, NumPy)
- Familiarity with machine learning frameworks (e.g., TensorFlow, PyTorch, Scikit-learn)
- Experience with databases (SQL, Influx)
- Knowledge of data warehousing and ETL processes
- Familiarity with tools like Hadoop, Spark, or Kafka
- Experience with cloud services such as AWS, Google Cloud, or Azure
- Understanding of software engineering principles and best practices
- Experience with version control systems (e.g., Git)
- Ability to design and implement efficient algorithms and solutions
- Demonstrated experience in deploying machine learning models to production
- Experience with data visualization tools and techniques
- Strong analytical and communication skills
- Ability to work collaboratively in a team environment
- Ability to communicate effectively, both orally and in writing
- A self-starter with the ability to work as part of a team in a fast paced environment with minimal supervision
- In addition, the following is considered not necessary but beneficial:
o Experience with Agile development practices
o Understanding of automation mechanical, electrical and control systems
o Understanding of machine operation, maintenance, service and troubleshooting
o Understanding of Machine Vision systems and solutions
o Understanding of PLCs and PLC communication
o Exposure and understanding of business intelligence
