10 Years Worth of Valuable Advice From a Data Engineer (Navigate the DataWorld)

10 Years Worth of Valuable Advice From a Data Engineer (Navigate the DataWorld)

I remember the first time I ever typed “SELECT * FROM” I was 18 and had no idea what I was doing. I clearly remember thinking, “Star” what?

I also remember the first database I ever dropped – accidentally (it was in development if you were wondering).

I also remember restoring it — sharpish.

I remember figuring out how to load CSV files — we still used those surprisingly — clean them up, removed white spaces and special characters, added a column and loaded it into a table. Amazed by it all.

Those were the early days.

It blows my mind what you can do with data. How many problems it can solve and how many companies sit on gold mines of the stuff, doing nothing with it.

I’ve been doing this data thing for well over 15 years. Actually, it’s more like 20, but I don’t count the missing 5 years. I was young and foolish, as youth often are.

Nowadays, I have a hunger for learning, a passion for data, and for figuring things out.

I love helping people and lifting people up.

I get a kick out of chugging through terabytes of data like it’s nothing.

I’ve seen a lot over the years, worked with incredible people, and learned a ton.

Here are the facts I’ve figured out and some advice and lessons I’ve learned along the way.

The Facts I’ve Learned

Every successful person who has ever worked in tech, be it a Developer, Data Engineer, Data Scientist, Data Analyst, seasoned veteran, or newbie has:

  • Felt like an imposter.
  • Spent hours figuring things out, even days.
  • Messed up and broke something badly, but learned from the mistake.
  • Had hard days, sad days, and tough days.
  • Felt like giving up.

The greats out there look like they are winning all the time.

The truth is they’re not.

  • They struggle, just like you.
  • They don’t know everything, just like you.
  • They Google and copy and paste code and rework it to make it fit, just like you.

The difference it they keep at it every day and make it look easy.
It’s okay to feel out of your depth, we all feel that way.

You build resilience by failing and realising nothing bad happens.

Lessons I picked up

Here of some lessons, I’ve picked up along the way, some lessons I learned the hard way. A few I learned from others and used to better myself. Most, I’d guess, are common sense, but useful nonetheless.

If you’ve just started or you’re a seasoned pro, hopefully, you find something useful here to use and learn from.

  1. Be willing to learn new things.
  2. Ignore the noise and focus on the fundamentals.
  3. You don’t need to know it all — you really don’t.
  4. Anyone can pick up and learn something if they are willing.
  5. SQL is a sledgehammer — learn to wield it.
  6. Read code written by experienced developers, programmers and data engineers.
  7. Be relentless in your routine but patient in your progress — things take time to sink in.
  8. You can’t learn everything by reading a book or watching a tutorial — you learn by doing.
  9. You’ll never know all the answers — as you figure things out, your experience grows.
  10. Ask forgiveness, not permission.
  11. There are those who can data model, and those who can’t — Learn to data model.
  12. Document your journey, and share it with others — you don’t know who is reading and who you can help or inspire.
  13. Simple is better than complicated — no exceptions.
  14. Data visualization is data’s special sauce — get the basics down.
  15. The answers are at your fingertips — find them.
  16. Small problems like to become big problems —solve problems early on.
  17. Avoid the hype — pick one cloud platform and learn it well.
  18. You don’t have to know Snowflake, Spark, Databricks, or “insert something else here” to be a data pro.
  19. If it feels like what you are doing is repetitive — automation is the key.
  20. Think through all the worst-case scenarios — there is always a worst-case scenario.
  21. Learn the art of problem-solving — the answers may be obvious but that doesn’t mean it’s the right way to do something.
  22. Always have a backup plan. — that means back up your databases often, and make a copy of the table when doing anything important — this is your get-out-of-jail-free card.
  23. Avoid DISTINCT to solve your duplication problems.
  24. Use the right tools for the job.
  25. Make friends with data scientists, they are crazy smart.
  26. Build your own side projects — just for fun.
  27. Write code every day.
  28. There is no such thing as a small schema change.
  29. Get good at Git — You will thank yourself later.
  30. Build strong interpersonal skills — make friends with people who use your data.
  31. A good team matters — find a team with a great leader.
  32. Ask questions — it’s okay to not know, don’t struggle alone ask for help.
  33. If you are in your comfort zone — You are not learning.
  34. Stay as up-to-date as you can with the data world — read blogs, LinkedIn posts or join a community online.
  35. Be organised. Find a process that works for you and stay on it.
  36. Taking a 15-minute walk away from your keyboard is an underrated form of problem-solving.

There you have it. I didn’t get to where I am alone. After writing this article it made me realize how far I have come. It made me realise how many people had a hand in shaping me as a person and lifting me up in my career.

It’s so easy to look at people’s LinkedIn profiles or blogs and think they have it all figured out. Truth is, they don’t. I don’t, that’s for sure. I still struggle, google, and ask questions. Things don’t always work out the way I plan them to in life and in my career, but if I have one skill that I value, it’s that I never give up. The answers and solutions are always out there in some shape or form.

If you are reading this because you are thinking of starting a career in data, know this: some days are tough, others not so much. Personally, I have more good days than bad ones, and in the data world, if you string enough good days together, it’s a pretty good career to get into.

Best of luck on your journey.


Thanks for reading! If you liked it, don’t forget to subscribe to my latest articles. Want to get in touch? feel free to find me on LinkedIn.