Client

Both the client and worker integrate Rust with Python through pyo3, a crate facilitating this foreign function interface.

User-Facing Library

The client defines a Python library that a user can import. The main entrypoint for this library is in client/src/torch/vtensor.rs, where the caravan module is defined with the Caravan class and the remote_call function.

The class is defined in pycaravan/init.rs, which uses connection/client_endpoint.rs to communicate with the distributor. After queueing, this class receives a list of tuples, each tuple containing a sender to an acquired worker machine and the GPU offset. For example, GPU 4 on worker machine 1 might be assigned as GPU 0 on the user's machine, so this offset represents the position on the worker machine.

See Peer Connections for details on how the client negotiates peer connections with workers and how bijection works.

Build the client with:

# from thava/
cd crates/client
maturin develop --release

# or, again from thava/
maturin develop --release --manifest-path crates/client/Cargo.toml

You can now import caravan and request GPUs.

Internal Library

Most bijection code is abstracted in the rtorch library, located at crates/client/src/rtorch. Install this library using:

cd crates/client/src/rtorch
pip install -e . # or uv pip install -e .

You must have rtorch installed for the same Python executable used to develop the main caravan client Python library.