At Bristol Myers Squibb, our employees often ask, “Who are you working for?”—a question that fuels collaboration, accountability, and urgency in our work. Our purpose-driven culture inspires us to discover, develop, and deliver innovative medicines to prevail over serious diseases. We offer uniquely interesting and meaningful work, opportunities for growth, and a supportive environment that values inclusion, wellbeing, flexibility, and comprehensive benefits. This is work that transforms the lives of patients, and the careers of those who do it.
The Applied AI Analyst II, Enabling Functions will help evolve Enabling Functions Analytics from static reporting toward intelligent, conversational, and reusable analytics products. The role combines practical AI research, context engineering, semantic modelling, and hands-on prototyping to connect governed enterprise data with large language models and modern analytics platforms. The individual will investigate how LLMs process context and generate responses, translate those learnings into reliable solution patterns, and build AI-enabled experiences using Claude, Databricks, Power BI, and related technologies. The role includes building text-to-SQL workflows, tool-using agents, and multi-agent systems that answer business questions reliably and transparently
Roles & Responsibilities
- Apply working knowledge of concepts and procedures to a variety of analytics and AI assignments, using judgement to solve problems with a moderate level of guidance.
- Own well-defined research, experimentation, and prototyping workstreams, documenting assumptions, decisions, results, and limitations.
- Research and test LLM behaviours, reasoning patterns, context-window usage, tool use, structured outputs, and failure modes, and convert findings into practical design guidance and reusable patterns for analytics use cases.
- Design and optimise system instructions, business context, examples, metadata, retrieval strategies, and tool interfaces so AI responses are grounded, relevant, and consistent. Manage context quality, freshness, and token efficiency.
- Build text-to-SQL workflows that translate natural-language questions into governed queries, grounded in certified semantic models and metric definitions, with validation steps that check generated SQL before execution.
- Design tool-using agents that call approved functions and data services (such as SQL execution, metric lookup, and visualization generation) with clearly defined tool schemas, permissions, and fallback behaviour.
- Design and prototype multi-agent workflows in which specialist agents (for example, query planning, SQL generation, validation, and insight narration) coordinate under an orchestrator, with clear handoffs, state management, and loop and cost limits.
- Prototype chat-with-data and chat-with-document experiences that let users explore governed business information in natural language, while preserving traceability to approved data and metric definitions.
- Design and improve business-ready semantic models covering grain, dimensions, measures, relationships, hierarchies, synonyms, descriptions, and certified definitions, ensuring the same logic supports Power BI, Databricks, and AI experiences.
- Evaluate when to use prompting, retrieval-augmented generation, tool calling, multi-agent workflows, or traditional analytics, and define fit-for-purpose architectures rather than applying LLMs where deterministic methods are more appropriate.
- Create test cases and evaluation datasets for SQL accuracy, tool-call correctness, agent task completion, groundedness, latency, and cost. Investigate hallucinations, retrieval or semantic failures, and agent loops, and recommend corrective actions.
- Build proof-of-value solutions using Python, SQL, APIs, Claude, and Databricks, integrating AI capabilities with existing analytics assets and, where appropriate, Power BI or lightweight application experiences.
- Break ambiguous business questions into clear problem statements, hypotheses, data needs, and solution options. Rapidly learn unfamiliar domains and communicate trade-offs to technical and non-technical stakeholders.
- Document patterns, reusable components, guardrails, and lessons learned, and contribute to internal standards for context engineering, semantic models, LLM evaluation, and responsible AI adoption.
- Partner with business analysts, BI developers, data engineers, AI/ML specialists, architects, and governance teams to move successful prototypes toward sustainable products.
- Balance innovation with responsible AI, data governance, security, privacy, quality, and production-readiness expectations, including human-in-the-loop controls for consequential actions.
Skills and competencies
- Strong analytical thinking, curiosity, and structured problem-solving, including comfort with ambiguous questions and incomplete information.
- Practical understanding of how large language models work, including tokenisation, embeddings, attention, context windows, inference, common failure modes, and the difference between model knowledge and supplied context.
- Hands-on expertise in prompt and context engineering, including instruction design, few-shot examples, grounding, retrieval-augmented generation, structured outputs, tool use, and context optimisation.
- Strong SQL workflow skills: writing and reviewing complex queries, designing text-to-SQL pipelines, enforcing read-only access and query guardrails, validating generated SQL against schema and metric definitions, and diagnosing incorrect joins, grain mismatches, and aggregation errors.
- Experience designing tool-using agents, including function and tool schema design, argument validation, error handling, retries, and permission boundaries.
- Working knowledge of multi-agent architectures such as orchestrator-worker, planner-executor, and specialist-agent patterns, including state management, handoffs between agents, termination conditions, and observability across agent steps.
- Hands-on experience with Anthropic Claude or comparable enterprise LLM platforms, including API-based integration, prompt orchestration, tool calling, and solution evaluation.
- Hands-on experience with Databricks, including notebooks, SQL Warehouses, governed data assets, and lakehouse-based analytics workflows. Exposure to Unity Catalog, AI/BI Genie, MLflow, or Mosaic AI is valuable.
- Experience designing and integrating REST APIs or equivalent service interfaces to connect LLMs, enterprise data platforms, and application experiences.
- Proficiency in Python and SQL for data exploration, AI prototyping, integration, evaluation, and automation.
- Experience with Git-based source control, branching, and pull-request practices, together with CI/CD pipelines for automated build, test, and deployment
- Hands-on understanding of AWS deployment patterns for AI and analytics solutions, including environment configuration, security, logging, monitoring, and release processes is a plus
- Experience designing or consuming semantic models, with strong understanding of business metrics, dimensional modelling, relationships, grain, hierarchies, synonyms, and reusable business definitions.
- Experience with vector search, agent frameworks, Model Context Protocol, LLM observability, or automated evaluation methods.
- Ability to design experiments and evaluation datasets and assess groundedness, relevance, consistency, completeness, latency, and cost.
- Ability to present complex information to non-technical stakeholders in a clear and concise manner, and to create relevant and engaging PowerPoint presentations.
- Awareness of responsible AI, privacy, cybersecurity, data governance, access control, and human-in-the-loop controls.
- Clear written and verbal communication, with the ability to translate technical AI concepts into practical business implications and document reusable engineering patterns.
- Ability to work collaboratively in cross-functional teams and manage multiple projects simultaneously.
Good to Have
- Practical machine learning knowledge, including supervised and unsupervised learning, feature engineering, model evaluation, and basic MLOps.
- Experience with Power BI, including semantic models, DAX, Power Query, row-level security, and integration of BI assets with AI-enabled analytics experiences.
- Familiarity with React or lightweight full-stack application development for creating user-facing AI experiences.
- Experience with agent orchestration frameworks and production monitoring of multi-agent systems.
- Experience working in regulated or highly governed enterprise environments.
Experience
- Bachelor's or master's degree in Computer Science, Artificial Intelligence, Data Science, Engineering
- Approximately 3–5 years of relevant experience in analytics, data science, AI engineering, software engineering, or a closely related field. Strong applied project work may substitute for some formal experience.
- Demonstrated experience solving analytical problems using Python and SQL, including building or reviewing text-to-SQL or query-generation workflows.
- Demonstrated hands-on delivery involving Claude or a comparable LLM, tool-using agents or multi-agent workflows, Databricks, APIs, Git/CI/CD, and deployment of an AI-enabled analytics prototype.
- Experience working with stakeholders to translate a business question into a testable analytical or AI solution.
- Experience producing clear technical documentation and communicating analytical results using compelling oral and written storytelling.
- Ability to handle, and maintain the confidentiality of, highly sensitive information.
We hire for skills and capabilities, not just credentials – if this role excites you, but doesn’t perfectly match your resume, we encourage you to apply anyway.
How We Work
Where you work matters – because collaboration, innovation and patient impact happen in many settings. Our roles are structured across four work models: site-essential, site-by-design, field-based and remote-by-design. The model assigned to this role is based on its core responsibilities. Learn more at https://careers.bms.com/ways-of-working.
Supporting People with Disabilities
BMS is dedicated to ensuring that people with disabilities can excel through a transparent recruitment process, reasonable workplace accommodations/adjustments and ongoing support in their roles. Applicants can request a reasonable workplace accommodation/adjustment prior to accepting a job offer. If you require reasonable accommodations/adjustments in completing this application, or in any part of the recruitment process, direct your inquiries to adastaffingsupport@bms.com. Visit careers.bms.com/eeo-accessibility to access our complete Equal Employment Opportunity statement.
Candidate Rights
BMS will consider qualified applicants with arrest and conviction records, pursuant to applicable laws in your area.
For roles based in Los Angeles County only: If you live in or expect to work from Los Angeles County if hired for this position, please visit this page for important additional information: https://careers.bms.com/california-residents/
For roles based in Washington state only: If you live in or expect to work from Washington State if hired for this position, please visit this page for important information about the Washington Fair Chance Act.
Data Protection
We will never request payments, financial information, or social security numbers during our application or recruitment process. Learn more about protecting yourself at https://careers.bms.com/fraud-protection.
Any data processed in connection with role applications will be treated in accordance with applicable data privacy policies and regulations.
If this posting is missing required information required by local law or incorrect, contact BMS at TAEnablement@bms.com with the Job Title and Requisition number. Do not send application-related inquiries to this email. To check your application status, please login to your Candidate Home Account.
R1607602 : Applied AI Analyst II