The Role

We at Razorpay are looking for a Data Solution Engineer to join our growing team. The hire will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up. The right candidate will be excited by the prospect of optimizing or even re-designing our company’s data architecture to support our next generation of products and data initiatives.

Your responsibilities include

• Expanding and optimizing Razorpay’s data and data pipeline architecture, and optimizing data flow.
• Designing, implementing, and operating stable and scalable solutions for moving data from production systems to analytical data platforms and user-facing applications
• Collaborating with business customers to understand requirements and implement analytical solutions in a fast-paced environment.
• Designing, creating, managing, and utilizing large datasets for business purposes.
• Contributing to high-level design with guidance, including functional modeling and module breakdown.
• Building and executing data modeling projects across various tech stacks.
• Challenging existing approaches and proposing innovative ways to process, model, and consume data, considering tech stack choices and design principles.
• Building and integrating robust data processing pipelines for enterprise-level business analytics.
• Incorporating automated monitoring, alerting, and self-healing features (restartability and graceful failures) into consumption pipelines.
• Translating business requirements into technical specifications, including facts, dimensions, filters, derivations, and aggregations.
• Supporting analytics teams in debugging data accuracy and related issues.
• Communicating effectively with engineering, product, and analytics teams to define key business questions and build datasets to answer those questions.
• Demonstrating a passion for working with large datasets, integrating them to address business questions, and driving change.

Essential technical skillsets include

• Experience with Enterprise Business Intelligence/Data platforms: Sizing, tuning, optimization, and system landscape integration in large-scale enterprise deployments.
• Programming proficiency: Preferably in Python.
• Data manipulation and modeling: Experience with Advanced SQL and DBT or other data modeling tools/frameworks.
• Software development methodologies: Good knowledge of Agile, SDLC/CICD practices, and relevant tools.
• Performance optimization: In-depth knowledge of performance tuning/optimizing data processing jobs and debugging time-consuming jobs.
• Data modeling expertise: Proven experience in developing conceptual, logical, and physical data models for EDW (enterprise data warehouse) and OLAP database solutions.
• Distributed systems understanding: A solid grasp of distributed systems principles.
• Large-scale data experience: Experience working extensively in a multi-petabyte DW environment.
• System engineering in product environments: Experience in engineering large-scale systems within a product development context
• Workflow Orchestration: Experience with tools like Apache Airflow for managing and scheduling complex data pipelines.

Good to have

• Streaming Technologies: A good understanding of streaming technologies like Kafka and Spark Streaming would be beneficial.
• Big Data Ecosystem Experience: Experience with Hadoop, MapReduce, Hive, and Spark, including Scala programming, would be advantageous.
• Proficiency in Java, Scala