CF CloudFrame Job Scanner Open Dashboard →
Verified Active Opening

Member of Technical Staff, AI

Agencywithin • Long Island City, New York

Job Description

<p><strong>About the Role </strong></p> <p>As a Member of Technical Staff focused on AI, you will design applied AI systems and build the products around them across frontend, backend, and data. End-to-end product ownership is central to the role. You will define the problem, decide what to build, ship it, evaluate the result, and maintain it after launch.</p> <p>Most of your technical work will involve LLM-powered applications, agent systems, and the evaluation and reliability systems required to run them in production.</p> <p><strong>The role is full lifecycle</strong></p> <ul> <li>Find product-market fit. Start with a small version, test it with users, and decide whether to continue, change direction, or stop.</li> <li>Improve what works. Once a product proves useful, test changes and invest in the versions that perform best.</li> <li>Expand and scale. Grow adoption across WITHIN and with clients, and make the system reliable under increased use.</li> <li>Maintain it. Monitor performance, resolve problems, and continue improving the product.</li> </ul> <p>You will set direction with business partners. They provide domain context and user needs; you decide how to scope, sequence, and develop the product.</p> <p><strong>How we work</strong></p> <ul> <li><strong>AI-native.</strong> We use coding assistants and agent workflows throughout development. Engineers are responsible for testing and reviewing the resulting work.</li> <li><strong>Bias to ship.</strong> Engineers work directly with business teams, make product and technical decisions, and release work in small increments.</li> <li><strong>Fast feedback loops, evidence over vibes.</strong> We observe how products perform and use feedback and product data to decide what to build next.</li> </ul> <p><strong>What we’re looking for</strong></p> <p>How you operate matters more than any specific technology:</p> <ul> <li>You have owned products end to end, from an ambiguous initial problem through launch and ongoing operation.</li> <li>You can decide what to test first, define success, and change course when results do not support the current plan.</li> <li>AI tools are already part of how you scope, build, test, and maintain software.</li> <li>You can choose between model behavior, deterministic software, and human review based on the problem.</li> <li>You work independently, communicate tradeoffs, and keep the people involved informed.</li> </ul> <p>Technical foundation:</p> <ul> <li>Strong Python skills and experience building production systems.</li> <li>Experience building production applications with LLM APIs such as OpenAI, Anthropic, or Vertex AI.</li> <li>Experience building agent workflows that use tools, call external systems, maintain state, and handle multi-step tasks.</li> <li>Experience designing evaluations for LLM-powered systems, including test cases, quality measures, regression testing, and review of production failures.</li> <li>Understanding of context management, token limits, multi-turn conversations, tool calling, structured outputs, prompt design, and common failure modes.</li> <li>Experience integrating external APIs and working with SQL.</li> <li>Experience tracing and debugging behavior across model calls, application code, tools, and external systems.</li> <li>Experience with GCP, AWS, or Azure.</li> <li>Ability to build the product around the AI components, including frontend, backend, and data. Experience with TypeScript, React, and backend services is a pl

Job Reference ID: CF-145655 • Posted on CloudFrame Job Scanner