Data Engineering

1 readers

1 users here now

Discussion on Data Engineering topics. Data pipelines, tools and technologies, databases and DBMS, best practices:

Rules:

Limited to data engineering, no general CS/programming posts.
No technical questions. Example: how to fix this bug in my code.
No marketing
No resumes, jobs
No PII

founded 1 year ago

MODERATORS

lucas50a@lemm.ee

Help Branching Out / Upgrading Skillset (lemm.ee)

submitted 1 year ago by starlord@lemm.ee to c/dataengineering@lemm.ee

5 comments fedilink hide all child comments

I've been a data (reporting) analyst for nearly a decade. I got my start writing complex SQL queries to develop reports and expanded into visualization (Tableau, Power BI). Lately I spent time as a data modeler designing DBs and helping create ETLs using ADF (though not putting hands on that process myself, I'm familiar enough with it to make it work).

I'm looking for my next opportunity and finding that my lack of knowledge in Python is creating a blocker. Also, other skills I don't have seem to crop up, like Spark, Hadoop, etc. It seems like the Data Engineer role has been folded into my Data Analyst role and I just can't compete any more.

Would anyone have any suggestions for paths I might take to remedy this? I can work with some online Python courses but I feel I'm not getting the full experience needed to really support the new needs being asked of those in my role. I'm hoping someone might have some suggestions for directions I might take to up-skill myself and be more prepared for the emerging needs of this changing industry.

top 5 comments

sorted by: hot top controversial new old

[–] vynlwombat@lemmy.world 2 points 1 year ago (1 children)

I think you might have better results if you look for BI engineering positions. That would be a better fit for your skill set. At some point data engineering sorta got split up into more specialized niches and based on your experience with dashboarding and ETLing, I think youre more in line with BI Engineering.

[–] starlord@lemm.ee 1 points 1 year ago

Thanks. I'll keep that in mind.

[–] Mamertine@lemmy.world 1 points 1 year ago (1 children)

Start applying.

If you want to learn Python, edX has a good course on it.

I'm a data engineer. I write Python once or twice a year. I'm terrible at it.

My shop has no spark or Hadoop anymore.

We want applicants who can write SQL really well and have experience modeling data. You can learn the tools easily if you have experience with similar tools

[–] starlord@lemm.ee 1 points 1 year ago (1 children)

Well hey, if you're looking for experience with SQL and data modeling, I'm available.

I'll look into edX, thanks for the suggestion!

[–] Mamertine@lemmy.world 2 points 1 year ago

A thought, we were hiring a data engineer. We saw a lot of resumes that had data engineer titles, but highlighted reporting skills not data engineering skills. If you're applying for a role, be sure your resume reflects that you have skills and accomplishments for the role you are applying for.