Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 21 hours
Course Outline
Introduction to Ollama Scaling
- Overview of Ollama’s architecture and key scaling factors
- Identifying common bottlenecks in multi-user setups
- Best practices for ensuring infrastructure readiness
Resource Allocation and GPU Optimization
- Strategies for maximizing CPU and GPU efficiency
- Considerations regarding memory and bandwidth usage
- Applying resource constraints at the container level
Deployment via Containers and Kubernetes
- Containerizing Ollama using Docker
- Executing Ollama within Kubernetes clusters
- Implementing load balancing and service discovery
Autoscaling and Batching
- Formulating autoscaling policies specific to Ollama
- Utilizing batch inference techniques to enhance throughput
- Balancing latency against throughput performance
Latency Optimization
- Profiling inference performance metrics
- Employing caching strategies and model warm-up techniques
- Minimizing I/O and communication overhead
Monitoring and Observability
- Integrating Prometheus for metric collection
- Developing dashboards using Grafana
- Establishing alerting mechanisms and incident response protocols for Ollama infrastructure
Cost Management and Scaling Strategies
- Implementing cost-aware GPU allocation
- Evaluating cloud versus on-premises deployment models
- Adopting strategies for sustainable scaling
Summary and Next Steps
Requirements
- Proficiency in Linux system administration
- Conceptual understanding of containerization and orchestration
- Background in deploying machine learning models
Target Audience
- DevOps Engineers
- ML Infrastructure Teams
- Site Reliability Engineers
Custom Corporate Training
Training solutions designed exclusively for businesses.
- Customized Content: We adapt the syllabus and practical exercises to the real goals and needs of your project.
- Flexible Schedule: Dates and times adapted to your team's agenda.
- Format: Online (live), In-company (at your offices), or Hybrid.
Price per private group, online live training, starting from 4350 € + VAT*
Contact us for an exact quote and to hear our latest promotions