Kafka

Technical Requirements: Technology->Cloud Platform->AWS Data Analytics->Amazon Managed Streaming for Apache Kafka Responsibilities: • Design, implement, and support Kafka-based streaming solutions for high-throughput, low-latency workloads. • Configure and manage topics, partitions, replication, retention, and compaction policies aligned to application needs. • Ensure cluster reliability and availability through capacity planning, upgrades, patching, and proactive maintenance. • Troubleshoot production issues including consumer lag, broker performance, rebalancing, and message delivery concerns. • Define and enforce standards for naming, ACLs, quotas, and multi-tenant usage to ensure safe platform adoption. • Implement monitoring and alerting for Kafka health, throughput, latency, and storage using metrics and logs. • Partner with engineering teams to guide producer/consumer configurations and best practices for scalability and resilience. • Drive performance tuning across brokers and clients, including partition strategy and batching/compression settings. • Document operational runbooks, architecture patterns, and onboarding guidelines for consistent usage across teams. Preferred Skills: Technology->Cloud Platform->AWS Data Analytics->Amazon Managed Streaming for Apache Kafka