>>110383114
Hey Cudadevanon, who owns the transport.cpp file in llama.cpp? The one generally found here when building from the source:
/home/user/llama.cpp/ggml/src/ggml-rpc/transport.cpp
I figured out if you change the
qia.cap.max_inline_data = 256;
value to
qia.cap.max_inline_data = 0;
it allows for the on-board 25G Broadcom nic (BCM57502) on the Gigabyte MC13-LE3-000 and other similar modern AM5 motherboards to use the RDMA transport when using the device as an RPC server! Without breaking anything else or defaulting to TCP! This modification isn't required if you're using this sort of node as the coordination node that's actually invoking llama-server \...
, but if you want it to act as an RPC server, it requires that inline value modification for it not to default to TCP.