There are many ways to become a data engineer. Especially in a field as relatively new as data engineering, guidance varies tremendously. That being said, throughout our years of experience, there are a few key lessons that we have learned when it comes to figuring out what makes a good data engineer.
Tip #1: Learn both on the job and at home
Let's get the basics out of the way - the best way to learn how to be a data engineer for most people is to both learn on the job and to learn at home. By reading about theory/practicing and implementing in the real world, you can reinforce and speed up your learning. (https://ablconnect.harvard.edu/authentic-learning).
Even if you can’t get a job in data engineering, try to find opportunities where there are analogous opportunities. For some jobs especially, that’s just not possible, but there are many analogous jobs that often provide data engineering opportunities. For example, if you are working as an analyst, try creating quality control unit tests with python when you have spare moments and leverage that into larger projects after obtaining some quick wins. If you’re a project manager or a secretary, try automating some features of your job like email reminders, form creation, etc. After creating a portfolio of these use cases, try applying for entry level data engineering positions.
A quality entry data engineer role will foster your growth, teach you essential skills, allow you to jump start your career and build your network, and help you develop your own personal brand.
As a junior data engineer, (as tough as this might sound especially in this economy) the salary should not be number one on your list when finding a job - opportunities to network and learn from mentors will give you plenty more chances to increase your pay in the future with interest.
Tip #2: Leave toxic data engineering jobs
While getting a quality data engineering job as quickly as possible should be the goal, it's important to be careful when accepting a job offer. Many companies that hire junior data engineers often make them work on rote tasks or overwhelm them with minimal support. Instead of providing an opportunity for growth, these toxic jobs can stunt your growth and development, teach bad habits, and burn you out early in your career.
Don't be afraid to leave a toxic job early - engineering is one of the fields where job hopping is more acceptable. When it comes to making career decisions, you have to do what's best for you.
Tip #3: Don't do the minimum
During our careers overseeing data engineering resources, one of the top things we've heard from poor performing data engineers is "our boss hasn't been clear with what they want" or "I'm waiting around to be given more work". As an engineer, there are an infinite number of skills and/or processes that could be improved. Poor performing data engineers will give the argument, "I'm not being paid to give 100%", but the motivation should come from what you can do to improve for YOU, not just the company.
Being given more responsibility early in a career comes with benefits in terms of learning new skills. Many career development frameworks will teach you to stand your ground and only do the work that is in your job descriptions, but as an early-career engineer, it is not the same situation. The more new skills, high pressure situations, and pipelines you are introduced to, the more reliable you will be later on in your career.
So yes, while you may be overworked for not enough money, once you master the technical skills that are required, you can apply for a new job more confidently. Impressing your more senior engineers will also do wonders for your network. As we mentioned above - job hopping is more acceptable for data engineering than it is for most fields. There's not as big of a risk when it comes to expanding your job scope.
Tip #4: Keep interviewing
It's easy to get complacent once you find a data engineering job that feels like home. What used to be exciting and novel gradually turns into comfortable. Senior devs who would hold your hand and provide tips slowly move on to other opportunities, and increasingly you find yourself becoming the person that shows the ropes to newer employees. That's a wonderful story, but only if the compensation is matching the progress.
We emphasized that salary shouldn't be the number one consideration when it comes to starting data engineering career, but at this point, you're no longer new. When it comes time to be recognized as a senior dev, you need to be aggressive to get the title and associated compensation range you deserve. In order to make sure you keep ahead of your career, you'll need to make sure that you're continuing to evaluate new opportunities, teaching yourself the new tools and development frameworks that are taking over the industry, and most importantly, continuing to interview.
By interviewing for higher positions at more reputable companies, you increase your options. Even if you're confident that your job is your dream job, it doesn't hurt to interview. It keeps your interviewing skills sharp, gives you motivation to remaster the basics and fundamentals, and continue to challenge yourself at home. When you're too comfortable, it limits the opportunity to grow. You might make the conscious decision that it's fine one day, but the important piece here is that you are making the conscious decision. For many they don't notice that they've lost the drive until it's too late, and then they learn a hard lesson when their company makes a hard decision. The more you interview, the more options that you have.
To stay ahead of the curve, check out dataengjobs.com to see the largest list of quality data engineering jobs refreshed daily!
Tip #5: Network
As you get older, your network and your skills will be your two most important assets. Your network will provide new opportunities when it comes to finding new jobs, collaboration opportunities, conferences, and more. Networking is a skill that we are also starting to learn - it's a challenge due to the amount of mental effort it takes to provide value to your community. It requires a lack of complacency - keeping on top of your skills and keeping track of your contacts.
----------------------------------------------------------------------------------------------------------------------
These are just a few tips and tricks for newer Data Engineers. We hope they were helpful, and make sure to check out dataengjobs.com for new blog posts!