is there any reason why I should use GGML or anything else over picrel? Porting pytorch models to onnx is just so much easier in comparison to porting to ggml but mostly because ggml lacks proper documentation.>INB4 its made my MicrosoftDon't care
onnx is a format if I am not wrong while ggml is a framework to run inference gguf is their current format. Yeah ggml sucks ass to write but setting up onnx can be pain in the ass and not all platforms are supported as well as ggml. I think kha-white/ocr was one thing that is a major pain in the ass to setup on onnx at least a year ago when I tried.
>>101547222ONNX is a format but I'm referring to ONNX runtime which is the inferance engine.from what I see Onnx runtime handles most of the heavy lifting whilst ggml requires you to create a bunch of different custom tensor operations, especially if you're porting from pytorch.I will agree that ONNX runtime was a pain to get working with CMAKE, they do seem to be using cmake as a build system but they're also using python to call the cmake. For now I'm just downloading the binaries and linking against them.
