How I Interview Data Engineers

Here’s what you want to do if you want to work with me

Photo by LinkedIn Sales Solutions on Unsplash

The market is brutal these days. Loads of people are looking for jobs, and those who already have jobs are unsure how long they'll keep them.

Now let me tell you something:

Hiring isn't stressful only for you as the candidate. It's stressful for the hiring manager, too.

And while I am not looking to add new data engineers to my team now, I want to help my future self by helping you.

As a hiring manager, I aim to get you on my team as quickly as possible. And the best way to do so is by being completely open and honest with you.

So, I won’t hold anything back. In this week's newsletter issue, I’m sharing anything you need if I ever interview you for a data engineer.

In the best-case scenario, you’ll read the whole piece here and learn what you need to know. And honestly, that’s more than enough for me to want you in my team.

This is what we are discussing this week:

• What I Am Looking For

• Digging Deep

• What Rounds Do You Have

• The Start

• What Excites You

• Standard Questions

• Closing

• After The Interview

Reading time: 11 minutes

---

What I Am Looking For

I am looking for a great balance of hard and soft skills. And while hard skills are essential, the soft side has more weight for me.

You may be the best data engineer in the world, but if you are an asshole, I don't want to work with you. On the other hand, you may lack a good portion of the skills I am looking for, but if you are a great person with great potential, I'll do my best to get you on board.

I am definitely not expecting you to have experience with the stack we use in my team, but I want you to know the tools you currently use. And even more importantly, I hope you are keen to learn and grow.

I once interviewed a candidate with an impressive background. They demonstrated proficiency with our tools and answered questions well. However, their constant negative remarks about previous teams raised concerns about their attitude and teamwork. Toxicity is contagious, and I quickly gave a hard NO to this person.

In contrast, in my web dev days, we hired somebody transitioning from a completely different industry. This person just went through an academy where they had a course on a tech set unrelated to ours. But they really wanted to learn. That was one of the most successful hiring stories in my life.

So, I am mainly hiring for attitude. Yet, our conversation will mainly be framed around technical topics. And let me be upfront with you — we'll get deep.

Digging Deep

Now, I have a lot of experience, not just in data but also in infrastructure, web, IoT and other areas. This allows me to talk to you about most of what you may have on your sleeve.

But remember, my idea is not to make you fail. I want to hire you, but to do so, I need a clear picture of how deeply you know your traits. And if you tell me you've got experience with a particular tech, I want to see what you mean. I want to know what to expect from you if we work together.

As an example, I once interviewed a candidate who implemented their entire ETL process using JavaScript. Intrigued, I delved deeper into their technical knowledge by asking about JavaScript fundamentals. We discussed things like the difference between var, let, and const, and the candidate’s understanding of variable hoisting.

And, even if I don't have experience with a specific tech, I'd be more than happy if you could explain that to me. At the end of the day, interviews are an excellent learning opportunity.

Suppose you say in the resume or the interview that you know something. I expect you to know it. I prefer to be surprised positively instead of negatively.

Is that how you approach interviews, or are you just listing as many things as possible? What are the pros and cons?

Enough theory! Let me give you the exact blueprint for my data engineering interviews.

What Rounds Do You Have

I want you to move swiftly through the stages. If you go through the whole process, the time from my message until your final “Yes” should be less than two weeks. We'd often try to squeeze everything within a week or so.

So, how do we proceed?

I hate HR rounds, and I know you hate them too. Although we have a great People team, I want to be the first person to contact you. I'll message you personally (no templates here) and invite you to the interview. I'll also provide some materials to help you learn more about the job and the company.

You usually start with a 60-90 Zoom call with me, where we discuss all sorts of stuff.

Then, I'll have a short discussion with my colleagues, and if we like you (also if you like me), we'll schedule a call with someone else on the team.

They do not know exactly what questions I asked you, so some may be repeated. The idea of the second round is not to have a more challenging interview but for both you and us to have a more objective view if we want to work together.

If everything on the first two rounds went fine, you have a third terse call with my manager. If you make it to this round, we like you and want to work with you. This call is purely for your own sake.

Here's the breakdown:

• One extended call with me

• One call with one of my teammates

• A brief skip-level intro call

Now, let's focus on my round.

The Start

Now, I don't like traditional interviews. I want you to have the best possible experience. My idea for the interview is to have a friendly conversation between two people who are passionate about their craft.

After the usual introduction, I’ll ask you what you know about the company and the position — people who have done their research shine here. Most people don't prepare, so I quickly tell them what the company does so you can decide if that's a product you want to work on. I'll also tell you about our processes and your potential day-to-day tasks.

You may have more questions about the data engineer job on my team. I'll be more than happy to answer any single one of them.

I'm essentially trying to sell you that position!

In this part, I may also ask you why you are looking for a new job and if your current team (if you have one) would try to keep you. This part depends on what you want to say about yourself.

Let's summarise:

• Tell me more about you, but keep in mind I've read your resume.

• Do you know anything about the company?

• Here's what data engineers are expected to do in my team…

• Why are you looking for a new job? Why with us?

• Would your team be okay with you leaving them?

What Excites You

After that intro, I'll usually ask you about a recent project that got you excited. Again, the idea here is to understand more about you. I want to know what assignments and tech spark joy in you.

As I said, I want to hire you and give you the opportunity to shine. So, no matter where our conversation goes, I'll often revisit your story about the project that excited you.

I'll hear what you have to say about that project. Then I'll make you talk about it and infect me with your excitement. I'll ask a ton of questions and dig deeper.

I want to understand how deeply you know what you do. If you have a shallow knowledge about things you like, you'll never become good enough to come up with your projects.

I am not looking for mediocre people here.

Let's summarise again:

• What exciting project have you worked on recently?

• What tech did you use for that project?

• Do you know how it works under the hood?

• How large was the team?

• What was your role in the group?

• Why did you make some specific design choices?

I know what you are thinking:

What if I don't have anything exciting? Do you have any standard questions?

Standard Questions

I believe a talented data engineer can always turn a tedious project into an exciting learning opportunity. Yet, sometimes, I want to know how closely you fit into our current stack.

I've split my questions into categories that change depending on our stack and your experience.

I mean, I wouldn't ask you any Snowflake questions if you don't have any experience with the tech. On the other hand, we are not using Redshift anymore, but if you have experience with the tech, I 100% want you to talk about it.

Let me give you some examples of those categorised questions.

SQL & Databases

Now, That's a category I definitely want you to be proficient in. As a data engineer, you need to know how to write efficient SQL and structure your databases in a scalable way. I also prefer it if you knew more about the internals of at least one engine.

Here are some questions I have asked in the past:

• What is the difference between OLAP and OLTP databases?

• Talk about different kinds of JOIN clauses, you know. When do you use them?

• How do you use the HAVING clause?

• What views are, and why would you use one instead of a table?

• Can you talk about primary and foreign keys?

• Can you design the relationship between books and readers for a library warehouse?

• How do you optimise queries?

• What data structures are being used for PostgreSQL indexes?

Python and General Programming

Another area I want to touch on is programming. Ideally, you'd have experience with Python, but any programming language would work.

• Talk about SOLID, DRY and other best practices.

• What does good code look like to you?

• Composition over inheritance.

• Ask about Python-specific things like decorators, iterators and generators

• What is the difference between a list and a tuple?

• What is the difference between append and extend?

• How is memory managed in Python?

• What is a Dataframe?

• In Spak, what is the difference between a data frame and RDD?

Data Stack Questions

In the best-case scenario, you'd have experience with Git, Singer, AWS, Snowflake and other tools we use. And yet again, that is usually not the case, but it's completely fine as long as you are fine with other soft and hard skills.

The list of questions here can become rather long, but here are some examples:

• Have you worked with Git or other version control systems?

• Do you prefer merging or rebasing? Why?

• How does your EL tool of choice work? How do you sync DB data?

• What is Docker, and how is it different from a virtual machine?

• How do you store and access your ML models?

• What AWS S3 storage classes do you know? How are they different?

• What is a virtual warehouse in Snowflake?

• What is Time Travel? How does it work, and how does it affect your costs?

Closing

As I said in the beginning, my interviews never feel like an interrogation. They really are in the form of a friendly conversation.

For example, if I ask you a question and you don't know the answer, I'll tell you what it is. Also, I'll occasionally pause and ask you if you have any new questions for me.

Photo by Surface on Unsplash

And again, interviews are perfect for meeting new people and exchanging knowledge. So, I like to finish with a question that helps me grow:

• How do you test your code?

• How do you stay updated and learn new things? What newsletters and websites do you follow?

• Tell me more about your dev environment. What OS, Editor and other tools do you use?

After The Interview

At the end of the interview, you'll have time to ask a few questions or say anything you want. I'll also have formed an opinion if I want to work with you, but I won't share that with you immediately. As I said, I first need to discuss your application with my colleagues.

So, we'll wrap it up, and if we like you, you'll get an invite for your second interview. If not, I'll message you to inform you we are not continuing the process with you. Here's where I'll always have some direct and actional feedback for you.

Either way, you'll know the result by the next day after your interview. I want you to have the best possible interview experience, and even if you don't pass, we can still be friends on LinkedIn. And who knows, maybe we can work together after a while!

Final Words

In this week's issue, you learned the intricacies of my data engineering interviews.

You explored the key attributes I look for in candidates, the importance of a balanced skill set, and the significance of a positive attitude.

Now that you've gained insights into what to expect take a proactive step in preparing for your next data engineering interview.

Reflect on your experiences and brush up on SQL and programming best practices. Consider how you articulate your project excitement.

Remember, interviews are not just evaluations but opportunities for mutual growth. Approach them with enthusiasm, and let your passion for data engineering shine!

---

Did you enjoy that piece? Follow me on LinkedIn for daily updates.