DirecciónThis is Adyen
Adyen provides payments, data, and financial products in a single solution for customers like Meta, Uber, H&M, and Microsoft - making us the financial technology platform of choice.
At Adyen, everything we do is engineered for ambition.
For our teams, we create an environment with opportunities for our people to succeed, backed by the culture and support to ensure they are enabled to truly own their careers.
We are motivated individuals who tackle unique technical challenges at scale and solve them as a team. Together, we deliver innovative and ethical solutions that help businesses achieve their ambitions faster.
Adyen seeks a Machine Learning Infrastructure Engineer to join our Generative AI team in Madrid. The team focuses on enhancing efficiency among different internal operational verticals through AI automation.
We specialize in creating a platform for Large Language Models (LLMs) and its downstream applications using Open Source software like HuggingFace or Langchain, and models such as Falcon, Llama or Mixtral as well as OpenAI or other proprietary LLMs.
Key responsibilities :
- Collaborate with the team to deploy LLMs for downstream tasks such as ticket routing (text classification), summarization, sentiment analysis, and question-answer retrieval.
- Develop and manage ETL pipelines using PySpark and Airflow for data preprocessing and model training. Organize the orchestration of pipelines for ETL and Machine Learning training to streamline model development.
- Automate the ML pipeline using MLOps tools and practices and optimize it for scalability and performance.
- Containerize applications and manage the Kubernetes deployments as well as the infrastructure needed to deploy LLMs internally;
from GPUs to vector databases and inference components.
- Develop observability best practices for the whole LLM infrastructure and build the internal framework which allows the team to monitor the LLM behavior to ensure their robustness under real life conditions.
- Design and implement APIs or frameworks to facilitate the seamless integration and usage of LLMs within various applications and services.
- Stay up to date with the latest advancements in MLOps tools and practices.
Qualifications :
- 5+ years of professional experience as a Machine Learning Engineer, DevOps or MLOps Engineer showing a clear understanding of the end-to-end Machine Learning lifecycle
- Strong software development skills, including : version control (e.g. Git and preferable on Gitlab), coding best practices, debugging, unit and integration testing.
- Proficient in Python. Knowledge of PySpark, Airflow, MLflow, Terraform, Docker, Kubernetes, Helm, Kustomize and ArgoCD.
- Proficiency with observability tools, such as : Prometheus, Logsearch, Kibana and Grafana.
- Knowledge of data pipelines and ETL processes to prepare and manage data for ML training and inference. As well as model development and deployment frameworks.
- Solid understanding of DevOps best practices and tools to automate software development and deployment processes, and CI / CD concepts and experience in implementing these practices.
- Ability to diagnose and resolve model performance, scalability, and deployment issues.
- Familiarity with monitoring tools to track model performance, resource utilization, and system health. Experience in logging and error monitoring for ML models and applications.
Desirable additional requirements :
- Experienced with Open-source Machine Learning frameworks like Huggingface Transformers.
- General LLMOps experience is a plus, including model deployment, monitoring, resources, and infrastructure management.
