Name: Rodrigo Alves Gonçalves
Location: São Paulo, Brazil.
Role: Senior Data Engineer
Expert at: Data Engineering, Google Cloud.
Favorite apps/tools: Airflow, BigQuery and Chat GPT.
Currently watching: The Last of Us
Currently reading: Designing data-intensive applications.
Building resilient and secure data pipelines is one of the most important concerns when you are a data engineer, but I think the biggest challenges are related to data quality. Maintaining a high-quality pipeline when data is constantly being processed and/or transformed is very challenging but it is what turns into valuable and strategic assets.
Streaming, data quality, orchestration frameworks and Data Lakehouse. Considering the growing interest of companies in leveraging their data to make decisions, enhance business processes and evolve their products, every day there is a new trend and as a data engineer it’s a very exciting moment. I can mention 4 trends that I want to learn more about:
Apache Airflow (orchestration), Great Expectations (Data quality), Kafka (streaming), BigQuery and Databricks (Lakehouse)
I love the flexibility and how warm and helpful people are. Even working with people from different parts of the world with different cultures, I feel like I’ve been working here for years.
I use organization tools like Notion or Google Keep. Establishing a routine and a method for organizing notes, drawings and documentation is very important to me.
I also love working while listening to music, it gives me the right rhythm depending on what I need at each moment. Sometimes I prefer a LoFi playlist to relax when I'm thinking of a solution and when I need more focus, I prefer more energizing music.
I could mention several people who somehow inspire me in different areas, but I would say that I admire people who dedicate their time to help others. People who really care about each other, whether it's sharing knowledge, dedicating their time to charity, or even in small day-to-day attitudes.