I'm an application developer who can write basic SELECT queries but wants to level up to proper data engineering -complex queries, data modelling, ETL pipelines, and working with data warehouses. The AI era runs on data and I want to be literate.
Plan for: Learn SQL and Data Engineering as a Software Developer
Accidental high billing from cloud platforms if large queries or pipelines are left running.
Set up strict billing budgets and alerts in GCP immediately after account creation. Pause or delete Snowflake warehouses when not in use.
Getting overwhelmed by the complexity of Airflow and Docker setup.
Start with a very simple local cron job or GitHub Actions first to understand scheduling, then migrate to Airflow using Astronomer's local CLI tools which simplify Docker setup.
Losing momentum due to the 3-5 hours/week constraint.
Protect your learning time. Break tasks into 30-minute chunks. It's okay if a step takes 2-3 weeks to complete at this pace.
Ready to make this plan yours?