When you deploy a large language model (LLM) for inference on Amazon SageMaker HyperPod, there’s a gap between when you request a pod and when it’s ready to serve traffic. This gap is dominated by two sequential downloads: the inference server container image from Amazon Elastic Container Registry (Amazon ECR), and the model weights from…
We have reviewed Apple’s new M5 Pro/M5 Max chips in the beginning of the year and the overall performance level is excellent, even though the smaller MacBook Pro 14 cannot really handle the M5 Max. Back then we were able to perform some additional benchmarks with the M5 max in the larger MacBook Pro 16…
