Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 14 hours
Course Outline
Deployment Options
- Hosted API versus self-hosted trade-offs
- On-premises, edge and air-gapped patterns
- Single model versus model portfolio
- Cost per decision comparison
Hardware Sizing and Capacity
- Memory footprint by parameter count
- CPU-only versus GPU acceleration
- Concurrency and throughput planning
- Headroom for task length growth
Serving Stacks
- vLLM and SGLang options
- llama.cpp and GGUF execution
- Ollama for local development
- Choosing based on decision workload shape
Quantisation Without Losing Calibration
- Precision formats and memory savings
- Measuring quality and calibration loss
- Per-tensor and importance-aware variants
- Acceptance testing after quantisation
Latency Engineering
- Median versus tail latency
- Latency growth with context length
- Warm start and model caching
- Batching multiple decision questions
Edge and Constrained Environments
- Small decision models on CPU
- Apple Silicon and embedded deployment
- Offline operation and update strategy
- Air-gapped weight distribution
Containerisation and Scaling
- Image design for model serving
- Volume and cache management
- Horizontal scaling and routing
- Resource limits and GPU sharing
Observability
- Request, latency and error metrics
- Confidence distribution monitoring
- Decision outcome and escalation tracking
- Alerting on calibration drift
Versioning and Rollout
- Model and adapter versioning
- Shadow and canary deployment
- Pinned model files for audit
- Rollback procedures
Security and Multi-Tenancy
- Authentication and rate limiting
- Tenant isolation for prompts and logs
- Network isolation and egress control
- Audit logging of decisions
Requirements
- Linux administration and container experience.
- Familiarity with model inference servers and GPU management.
- Basic understanding of REST APIs, load balancing and monitoring.
Audience
- MLOps and platform engineers running inference infrastructure
- SRE teams supporting AI services
- AI engineers deploying decision models on premises
- Architects designing air-gapped or regulated AI deployments
Custom Corporate Training
Training solutions designed exclusively for businesses.
- Customized Content: We adapt the syllabus and practical exercises to the real goals and needs of your project.
- Flexible Schedule: Dates and times adapted to your team's agenda.
- Format: Online (live), In-company (at your offices), or Hybrid.
Price per private group, online live training, starting from 2900 € + VAT*
Contact us for an exact quote and to hear our latest promotions
