Salary
Astronomer is the commercial developer of Apache Airflow, a community-driven open-source tool that’s leading the market in data orchestration. We’re a globally-distributed and rapidly growing venture-backed team of learners, innovators and collaborators. Our mission is to build an Enterprise-grade product that makes it easy for data teams at Fortune 500’s and startups alike to adopt Apache Airflow. As a member of our team, you will be at the forefront of the industry as we strive to make Apache Airflow the de-facto standard in data orchestration. Cloud Platform Engineers at Astronomer develop the service infrastructure, application, operators, and API layers for Astronomer’s managed Airflow cloud solution. They are responsible for conceiving, designing, and implementing key components and interfaces of the Astronomer Cloud and its data orchestration management system. These services deploy, control, and observe all the instances of Airflow running within a customer’s network and data plane. Responsibilities Design and implement the Astronomer Cloud and its API, a highly available, scalable, and extensible service Solve challenging and complex technical problems, related to distributed systems, security, networks, data pipelines, and multi-cloud infrastructure. Leverage cloud computing infrastructure including third-party APIs and services Build Kubernetes infrastructure to effectively and efficiently manage large scale distributed deployments Develop UI/UX that simplifies customer workflows Research and innovate new technologies Work closely with the open source community and Airflow committers Fundamentals Distributed systems: database system implementation and internals, distributed storage structure, query languages, caching layer architecture, cluster computing Distributed computing theory: distributed job execution, concurrency, shared-memory model, message-passing model, complexity measures, election algorithms Networks: protocol functionality (TCP/IP, HTTP), RPC-like connectors, message queues Data engineering fundamentals: Scheduling theory and scheduler implementation, virtualization and containerization Key Experience Built entire or significant portions of large, complex SaaS infrastructure operating at scale (e.g., hundreds of thousands of users, thousands of customers, tens of thousands of jobs executing simultaneously)… Developed greenfield cloud-based systems with service-oriented architectures, and/or added new features or services to a large complex system with performance constraints… Built tools (e.g., internal APIs, debuggers, etc.) for streamlining internal development and external deployment of new services and features and enhancing developer productivity… Designed services for resiliency and high availability (e.g., implemented replicated storage and multi-homing in production deployments)... Been a top contributor or senior engineer building large, complex distributed systems, either proprietary or open source (e.g., Apache Kafka, Spark, Postgres, etc.)... You've likely received some formal training in computer science in the form of a bachelors, masters, or PhD degree At Astronomer, we value diversity. We are an equal opportunity employer: we do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. Astronomer is a remote-first company.
https%3A%2F%2Fremotive.io%2Fremote-jobs%2Fsoftware-dev%2Fcloud-platform-engineer-689987