Sr. Principal Engineer, Data
Acadia Healthcare
- Franklin, TN
- Permanent
- Full-time
- Design, implement, and optimize cloud data architecture including databases, schemas, tables, views, clusters in cloud data platform tech stack
- Develop end-to-end data pipelines that from source systems ingestion to data warehouse or data lakes and to data mart, eventually service BI and AI/ML for visualization and analytical needs
- Create curated data assets with access and governance in place for analytical workloads
- Monitor data pipelines, infrastructure health metrics. Automate for fault tolerant and self-healing, optimize for maximal throughput and performance
- Automation and integration of data quality components in the data pipelines.
- Modernize data assets and associated pipelines from legacy tech stack into new data platform
- Set, document and maintain standards for data assets inventory, meta data management and data lineage
- Research and evaluate new data management tools and approaches for potential integration
- Coach and mentor junior data engineers in data modeling, pipeline development, troubleshooting issues
- Complies with organizational policies, procedures, performance improvement initiatives and maintains organizational and industry policies regarding confidentiality
- Communicate clearly and effectively to person(s) receiving services and their family members, guests, and other members of the health care team
- Develops constructive and cooperative working relationships with others and maintains them over time
- Encourages and builds mutual trust, respect, and cooperation among team members
- Maintains regular and predictable attendance
- BS/MS degree in Computer Science, Engineering or equivalent field
- 7+ years hands-on engineering and administration experience in data engineering on cloud platforms (Azure preferred) with RDBMS such as SQL Server, PostgreSQL, MySQL, with Snowflake experience as must-have.
- 3+ years of experience with Apache Spark/Databricks and Azure Synapse.
- Expert knowledge of SQL and experience with Spark, Python for data transformation/processing
- Experience building and optimizing data pipelines at scale with orchestration tools like AirFlow and observability tools like ELK
- Excellent communication skills collaborating cross-functionally with stakeholders
- Self-directed and passionate about keeping up with latest innovations in the data space
- API development experience is a plus