this post was submitted on 31 Jul 2023
3 points (58.8% liked)
Machine Learning
1742 readers
1 users here now
founded 4 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
How do people use the HTTP interface in production? I'm curious because I found it necessary to use a queue in production in order to handle many requests and smoothly scale up workers, yet every containerization approach I've seen uses this synchronous HTTP based approach. Are there any standard containerization methods that rely on queues instead ?