Dynamic Batcher: a Flexible Batch Processing Daemon

Release:

v1.0.6.1

Overview

dynamic-batcher is a request handling tool to gather multiple requests together for batch processing. It is designed to handle model infereneces using GPU simultaneously, as parallel.

How-to Guide

• Getting Started
  • Quickstart
    • Gathering API requests as one batch
  • Command-line Interface
    • Command-line Interface to launch BatchProcessor
      • Args
      • Example

Modules

• dynamic_batcher package
  • BatchProcessor
    • BatchProcessor.delay
    • BatchProcessor.batch_size
    • BatchProcessor.batch_time
    • BatchProcessor.start_daemon
  • DynamicBatcher
    • DynamicBatcher.delay
    • DynamicBatcher.timeout
    • DynamicBatcher.asend
  • PendingRequestStream
    • PendingRequestStream.consumer
    • PendingRequestStream.message_id
    • PendingRequestStream.time_since_delivered
    • PendingRequestStream.times_delivered
  • ResponseStream
    • ResponseStream.body
    • ResponseStream.stream_id
  • Submodules
    • dynamic_batcher.batcher module
    • dynamic_batcher.logger module
    • dynamic_batcher.redis_engine module
    • dynamic_batcher.types module
    • dynamic_batcher.validate module

Indices and tables

• Index

• Module Index

• Search Page