Boost Your Data Team: 7 Practical Steps to Crush Skill Gaps Now
Learn actionable strategies to transform your data team from mediocre into a powerhouse of efficiency and innovation
Greetings, curious reader,
Data engineering is a rich and dynamic field. New tools and concepts pop up every year. Keeping up with the trends is not easy having in mind how overworked you and your team are.
The truth is that most innovation steps on existing concepts. If you are a data engineering leader, a big part of your role is to ensure your team has the necessary background to learn these new concepts quickly.
The truth, though, is that learning is not a one-off project. It's a marathon.
This guide will walk you through my strategy to upskill my DataOps team. You'll learn how to:
Identify crucial skills for modern data engineering
Assess your team's current capabilities
Create a personalised learning path for each team member
Foster a culture of continuous learning and growth By the end of this article, you'll have an actionable plan to transform your team from good to great. Let's get started!
Reading time: 13 minutes
The Problem: Stagnant Skills in a Dynamic Field
I've had this problem. I was stuck to classic ETL with data lakes and slow data processing.
Somebody told me about a startup called Snowflake and how they use S3 to separate storage from computing. I was sure this was crappy technology. Our stack was fine and worked the way a data platform is meant to work.
A couple of years later, I stumbled upon Snowflake again at my current company. We've been using them for more than 5 years now.
The data landscape changes faster than many are willing to admit. Your favourite tools and practices might have been working for a long time. Yet, it only takes a blink to find yourself in the middle of:
Technical debt
Failing pipelines
Delayed projects
Let's break down why this happens:
Changing business demands: Your organisation changes, and so do its data needs. An approach that worked for years may not cut it anymore.
Skill decay: Without constant practice and updating, you can easily forget skills you worked hard for.
Lack of structured learning: Many teams rely on ad-hoc learning. This leads to knowledge gaps and inconsistent skill levels.
Time constraints: With tight deadlines and heavy workloads, finding time for learning often gets deprioritised.
You can't just send your team to a conference. You can't expect them to magically upskill overnight. You need a systematic, ongoing approach to learning and development.
The Solution: Strategic Upskilling
Let's talk about strategic upskilling. This is a laser-focused approach to levelling up your team. Here's why it works:
It's tailored to your team's needs: No one-size-fits-all solutions here. You create a custom plan for your team.
It's ongoing, not a one-off event: Learning becomes part of your team's DNA, not just an annual checkbox.
It aligns with your business goals: The skills you develop directly impact your organisation's success.
Just check this out:
According to the World Economic Forum, an average of 66% of employers report a return on investment within one year of implementing upskilling programs for their employees.
The Workplace Learning Report for 2024 revealed that companies with solid learning cultures see 57% higher retention rates.
When I say strategic upskilling, I'm not only talking about technical skills. It's a holistic approach that covers:
Technical proficiency: Mastering the tools and technologies of modern data engineering.
Soft skills: Enhancing communication, problem-solving, and collaboration abilities.
Business acumen: Understanding how data engineering impacts the broader business context.
Adaptability: Developing the ability to quickly learn and adapt to new technologies.
Leadership: Nurturing future leaders within your team.
The 7-Step Upskilling Strategy
Now, let me tell you about the strategy I use. This 7-step approach does wonders and can transform how you think about team development.
Step #1: Map Your Skill Territory
First things first. You need to know where you are before finding where to go.
Create a list of must-have skills for your team. Don't just think about today's needs. It's about future-proofing your team.
Here's an example of how to create your skill map:
Technical skills
Programming languages: Python, SQL, Scala
Big data technologies: Hadoop, Spark, Kafka
Cloud platforms: AWS, Google Cloud, Azure
Data warehousing: Snowflake, BigQuery, Redshift
ETL tools: Apache Airflow, Luigi, Dagster
Version control: Git, GitHub
Soft skills
Communication: Written and verbal
Problem-solving: Analytical thinking, debugging
Collaboration: Teamwork, knowledge sharing
Time management: Prioritisation, meeting deadlines
Adaptability: Learning new technologies quickly
Emerging technologies
Real-time data processing
Data governance and security
DataOps and MLOps
Data visualisation
Don't just brainstorm this list yourself. Involve your team leads, consult with other departments, and look at job postings for senior data engineering roles. Think about your future plans. This gives you a comprehensive view of the skills landscape.
Step #2: Create Your Skill Matrix
Now, it's time to get organised. Build a matrix. Skills on one axis and team members on the other.
Here's how to create and use your skill matrix:
List all team members down the left side of your matrix.
List all the skills you identified in Step 1.
Rate each team member against each skill using this scale:
1 = Novice (Basic understanding, needs significant support)
2 = Intermediate (Can work independently on most tasks)
3 = Expert (Can teach others, innovates in this area)
Color-code your matrix for easy visual analysis:
Red for 1 (Novice)
Yellow for 2 (Intermediate)
Green for 3 (Expert)
Analyse your matrix:
Look for skill gaps: Areas where most of your team scores low
Identify experts: Team members who can mentor others
Spot growth opportunities: Skills where individuals are close to levelling up
This matrix gives you a bird's-eye view of your team's capabilities. It also highlights skill gaps and areas for improvement.
Step #3: Have Heart-to-Hearts with Your Team
Don't assume. Ask. Set up one-on-one chats with your team members. These aren't performance reviews. They're career development discussions.
Beware! This might seem a bit like a typical HR conversation. Only take this step if you are genuinely interested in your people's development.
Here's how to structure these conversations:
Start with their goals:
Where do they see themselves in 1 year? 5 years?
What part of their job do they enjoy most?
What would it be if they could change one thing about their role?
Discuss skills:
What skills do they want to develop?
Are there areas where they feel they're falling behind?
What skills do they think will be crucial in the future?
Explore passions:
What do they enjoy outside of work?
Are there hobbies or interests that could translate to valuable skills?
Talk about learning styles:
How do they prefer to learn? (Online courses, books, hands-on projects)
Do they enjoy teaching others?
What's been their most successful learning experience in the past?
Discuss barriers:
What's holding them back from learning new skills?
How can you, as a leader, help remove these barriers?
Listen more than you talk. You might be surprised by what you learn.
Step #4: Block Learning Time
Learning takes time. Make sure your team has it. This step is crucial. Without dedicated time, learning becomes an afterthought.
Here's how to make learning time a reality:
Make it official:
Ask each team member to block 1-2 hours weekly for learning.
This time should be treated like any other important meeting.
Lead by example:
Block your own learning time.
Share what you're learning with the team.
Protect this time:
Don't schedule over it.
If a crucial meeting must happen during learning time, ensure it's rescheduled and not cancelled.
Make it flexible:
Allow team members to choose when their learning time occurs.
Some might prefer early mornings, others late afternoons.
Provide resources:
Ensure your team has access to necessary learning materials.
This could include online course subscriptions, books, or conference attendance.
Step #5: Launch Bi-Weekly Training Sessions
Knowledge sharing is critical. Organise bi-weekly training sessions. These sessions are for you to teach your team skills you want them to learn.
Here's how to make these sessions effective:
Choose diverse topics:
New technologies
Best practices
Case studies from recent projects
Soft skills like communication or time management
Make it interactive:
Encourage questions and discussions.
Use tools like polls or quizzes to engage participants.
Keep it relevant:
Tie topics to current projects or challenges when possible.
Ask for topic suggestions from the team.
These sessions are collaborative learning experiences. Encourage debate, questions, and idea-sharing.
Step #6: Start Show and Tell
Let your team shine. Every two weeks, ask a team member to present.
Aside from sharing knowledge, this helps them build confidence and reinforce learning.
Here's how to structure your show and tell:
Keep it short:
Aim for 15-20 minute presentations.
Leave time for questions and discussion.
Vary the content:
Share a project they're working on
Teach a new skill they've learned
Discuss an interesting article or book
Demo a new tool or technology
Encourage creativity:
Let presenters choose their format (slides, live demo, whiteboard session, a workshop)
Welcome unconventional approaches
Make it voluntary... mostly:
Encourage everyone to present at least once a month
But don't force it if someone is genuinely uncomfortable
The goal isn't perfection. This is about sharing knowledge, building confidence, and fostering a culture of continuous learning.
Step #7: Monitor and Adjust
Progress isn't always linear. You need to keep a close eye on how your upskilling efforts are progressing. Be ready to adjust your approach based on what you observe.
Here's how to effectively monitor and adjust your upskilling strategy:
Regular check-ins:
Monthly one-on-ones to discuss learning progress
Quarterly team-wide reviews of the upskilling program
Measure progress:
Update your skill matrix every quarter
Track completion of online courses or certifications
Monitor participation in training sessions and show-and-tell
Gather feedback:
Anonymous surveys to get honest input
Open discussions in team meetings about what's working and what's not
Face-to-face discussions
Look for real-world impact:
Are new skills being applied to current projects?
Has productivity or quality improved?
Are there fewer blockers or dependencies?
Stay flexible:
Be ready to pivot if certain approaches aren't working
Remain open to new learning opportunities or technologies
Celebrate wins:
Recognise team members who've made significant progress
Share success stories with the broader organisation
Remember, upskilling is a journey, not a destination. Your strategy should evolve as your team grows and as the field of data engineering changes.
Real-World Success: The Power of Open-Minded Upskilling
Let me tell you a story. I had a brilliant data engineer on my team. Let's call her Camille.
Camille expressed interest in analytics engineering. We worked together to develop these skills.
A year and a half later, she wanted to do full-time analytics engineering. We couldn't offer that opportunity internally. She left and became a lead analytics engineer at another fintech.
During these 18 months, Camille was practising her learning. She brought a ton of valuable innovation to the team.
The moral? Sometimes, upskilling means letting go. But it also means:
Fostering growth and building lasting relationships
Creating ambassadors for your team in the wider industry
Developing a reputation as a great place to grow and learn
We're still in touch. And who knows? We might work together again one day.
Your Next Steps: From Reading to Action
You've got the blueprint. Now it's time to act. Here's what you need to do in the next 30 days:
Week 1
1. Create your skill list and matrix
Involve your team in brainstorming necessary skills
Complete an initial assessment of your team's current skills
2. Schedule one-on-ones with your team
Block out time for in-depth career discussions
Prepare questions to guide these conversations
Week 2
3. Block out learning time in everyone's calendar
Make this a recurring event
Communicate the importance of this time to your team and other stakeholders
4. Plan your first training session
Choose a topic relevant to your team's current challenges
Identify a team member to lead the session
Week 3
5. Launch your first show-and-tell session
Choose a volunteer or nominate someone to present
Provide guidelines and support for the presenter
6. Start implementing learnings
Encourage team members to apply new skills to current projects
Be open to suggestions for process improvements based on new knowledge
Week 4
7. Review and adjust
Gather initial feedback on the upskilling initiatives
Make necessary adjustments to your approach
8. Plan for the long term
Set quarterly goals for team skill development
Align these goals with upcoming projects and business objectives
Making the Business Case for Upskilling
As a data engineering leader, you might need to justify your upskilling efforts to upper management. Here's how to make a compelling case:
1. Link to business goals
Show how improved skills directly contribute to key business objectives.
Example: "By upskilling in real-time data processing, we can deliver insights 30% faster, allowing quicker decision-making."
2. Highlight cost savings
Demonstrate how upskilling can reduce the need for external consultants or new hires.
Example: "Developing in-house expertise in cloud platforms can save us £100,000 annually in consultant fees."
3. Emphasise retention
Explain how learning opportunities improve employee satisfaction and retention.
Example: "Companies with strong learning cultures have 30-50% higher retention rates."
4. Showcase innovation potential:
Illustrate how new skills can lead to innovative solutions.
Example: "Our team's new machine learning skills led to a predictive maintenance model, reducing downtime by 15%."
Want to learn more about business cases? Read this exceptional article I wrote.
If you enjoyed the newsletter so far, please show some love on LinkedIn or forward to your friends. It really does help!
Final Thoughts
Steering the wheel of your team's learning might be challenging. But with this strategy, you're well-equipped for the journey.
Now, it's important to remember this:
You are responsible for setting the expectations. But your team members are responsible for their learning.
If somebody refuses to keep up with the team standards, it's their problem. Your role is to set expectations and support all efforts.
It's also important to mention that goals change with time. A priority today might be irrelevant tomorrow. It might feel repetitive, but I prefer asking people often if their goals are still valid.
In my experience, people tend to stick to a goal because they've spent too much time working on it. However, pivoting when your priorities change is much better than wasting more time.
Summary
Let's recap the key points:
Skill mapping: Know your territory before you set out.
Skill matrix: Get a bird's-eye view of your team's capabilities.
Heart-to-hearts: Listen more than you talk. Your team might surprise you.
Learning time: Block it, protect it, lead by example.
Bi-weekly training: Share knowledge and foster debate.
Show and tell: Let your team shine. It's not about perfection.
Monitor and adjust: Stay flexible. Upskilling is a journey, not a destination.
Upskilling is about growing your team holistically. Sometimes, it means letting go of a star player. But more often, it means building a team that's adaptable, innovative, and always learning.
Start with your skill map. Then, have those crucial conversations with your team. Before you know it, you'll be leading a data engineering dream team.
Until next time,
Yordan
Be an Ambassador
Did you know? I wrote an extensive Snowflake learning guide. And you can have this for free!
You only need to share Data Gibberish with 5 friends or coworkers and ask them to subscribe for free. As a bonus, you will also get 3 months of Data Gibberish Pro subscription.
Join the Community
I believe that weekly newsletters were insufficient. A group of leaders and I launched an exclusive Discord community where you can dive into vibrant discussions on software and data engineering, leadership, and the creator economy.
Join today, and let's supercharge our professional journeys together!
How Am I Doing?
I love hearing you. How am I doing with Data Gibberish? Is there anything you’d like to see more or less? Which aspects of the newsletter do you enjoy the most?
Use the links below, or even better, hit reply and say “Hello”. Be honest!
