Dynamic Batcher: a Flexible Batch Processing Daemon
- Release:
v1.0.6.1
Overview
dynamic-batcher
is a request handling tool to gather multiple requests together for batch processing.
It is designed to handle model infereneces using GPU simultaneously, as parallel.