Data Engineer
OpenSearch Engineer / AWS & Data Pipelines
Удалённоfulltime
Откликнуться на сайте работодателяОписание вакансии
About Us
Ruby Labs is a leading tech company that creates and operates innovative consumer products. We offer a diverse range of opportunities across the health, education, and entertainment industries. Our innovative teams are driving the future of consumer-led products, and we're always looking for passionate individuals to join us. Learn more about our story at: https://rubylabs.com/about\-us/
About The Role
We are building a high-scale payment orchestration platform that processes millions of financial transactions every month across multiple providers, regions, and products.
We are looking for an experienced
OpenSearch Enginee
r to own and scale our AWS OpenSearch infrastructure and the pipelines that synchronize data from PostgreSQL.
You will be responsible for search architecture, data ingestion, performance, reliability, reconciliation, and recovery. The role combines search infrastructure, data engineering, and production operations.
Key Responsibilities
OpenSearch
- Own and operate AWS OpenSearch Service in production.
- Design indices, mappings, analyzers, templates, aliases, shards, and retention policies.
- Optimize search queries, aggregations, filtering, pagination, and bulk indexing.
- Investigate slow queries, indexing bottlenecks, hot shards, memory pressure, and rejected requests.
- Plan safe upgrades, migrations, reindexing, and capacity scaling.
PostgreSQL-to-OpenSearch pipelines
- Build and maintain reliable pipelines that synchronize PostgreSQL data with OpenSearch.
- Support inserts, updates, deletes, backfills, and full reindexing.
- Use AWS-native services such as AWS DMS, OpenSearch Ingestion, Lambda, SQS, S3, ECS, or EKS where appropriate.
- Implement retries, idempotency, deduplication, replay, dead-letter handling, and failure recovery.
- Monitor ingestion lag, throughput, failed records, and data freshness.
- Build reconciliation processes to detect and repair missing, stale, or inconsistent documents.
- Ensure ingestion does not negatively affect the transactional PostgreSQL workload.
Reliability and operations
- Monitor cluster health, search latency, indexing latency, storage, CPU, JVM pressure, and shard distribution.
- Build CloudWatch dashboards, alerts, and operational runbooks.
- Design snapshot, restore, backup, and disaster-recovery procedures.
- Participate in production incident response and root-cause analysis.
- Implement secure access using IAM, role-based access control, encryption, and least-privilege principles.
Collaboration
- Work closely with backend, database, infrastructure, and product teams.
- Help design efficient OpenSearch document models for payments, customers, subscriptions, and transactions.
- Define standards for searchable fields, schema changes, and index versioning.
- Document architecture, data flows, and recovery procedures.
Qualifications
- Strong production experience with OpenSearch or Elasticsearch.
- Hands-on experience with AWS OpenSearch Service.
- Experience building data pipelines between PostgreSQL and OpenSearch.
- Strong understanding of:
- Index and shard design
- Mappings and analyzers
- OpenSearch Query DSL
- Aggregations, sorting, filtering, and pagination
- Bulk indexing and cluster performance
- Data consistency, retries, idempotency, and replay
- Experience with large datasets and high indexing or query volumes.
- Experience performing backfills and reindexing safely.
- Strong troubleshooting and production-operations skills.
Nice to have
- Experience with AWS DMS or OpenSearch Ingestion.
- Experience with Lambda, SQS, S3, ECS, EKS, or Terraform.
- Experience building custom ingestion services in Go, Java, or Python.
- Experience in payments, billing, fintech, or subscription systems.
- Experience with multi-tenant or high-volume search platforms.
- Familiarity with ClickHouse, Tinybird, or similar analytical systems.
- AWS certifications.
Location
Ruby Labs operates within the CET (Central European Time) zone. Applicants from any country are welcome to apply for the position as long as they are located within approximately ± 4 hours of CET. This ensures optimal collaboration and communication during working hours.
Benefits
Discover the perks of being part of our vibrant team! We offer:
- Remote Work Environment: Embrace the freedom to work from anywhere, anytime, promoting a healthy work-life balance.
- Unlimited PTO: Enjoy unlimited paid time off to recharge and prioritize your well-being, without counting days.
- Paid National Holidays: Celebrate and relax on national holidays with paid time off to unwind and recharge.
- Company-provided MacBook: Experience seamless productivity with top-notch Apple MacBooks provided to all employees who need them.
- Flexible Independent Contractor Agreement: Unlock the benefits of flexibility, autonomy, and entrepreneurial opportunities. Benefit from tax advantages, networking opportunities, reduced employment obligations, and the freedom to work from anywhere. Read more about it here: https://docs.google.com/document/d/1nkrN76JlZkbKj9WSOhlT1\_mni\_CZeDkHdwfIjPXVwvk/preview?tab\=t.0\#heading\=h.ndsdl4wapxtt
Be part of our fast-growing team and seize this excellent opportunity for personal and professional growth!
Interview Process
After submitting your application, we conduct a thorough review which typically takes 3 to 5 days, but may occasionally take longer due to the volume of applications received. If we see a potential fit, we proceed with the following steps:
- Recruiter Screening (40 minutes)
- Technical Interview (60 minutes)
- Final Interview (60 minutes)
Life at Ruby Labs
At Ruby Labs, we move fast, aim high, and expect the same from our team. We’re not here to play small—we’re here to build, grow, and win. That means we look for people who are ambitious, driven, and ready to give their best every single day.
This is a place for individuals who thrive under pressure, embrace challenges, and see opportunity in every obstacle. If you’re hungry to achieve, motivated by impact, and want to grow at the speed of your own ambition, Ruby Labs offers the platform to make it happen.
Here, effort is matched with reward. We recognize those who go all in and deliver results, and we create space for people who want more—more responsibility, more growth, and more success.
Это сохранённая копия объявления, опубликованного в другом месте. Вакансии снимают без предупреждения — перед откликом проверьте сайт работодателя. mentors.coach не является нанимающей стороной.