AMD Support

[userbox020]

Hello,
Any plan to add AMD support, I and some friends can help

[AlpinDale]

Hi, AMD is technically supported but I don't have the hardware to run tests or seriously develop for it.

[userbox020]

so maybe i could use ooba text generation webui environment and install aphrodite engine inside the ooba enviroment?
going to try that and see if it works

[AlpinDale]

I will note that the support is currently extended to MI200+ AMD GPUs, so consumer ones are unsupported. I was waiting on upstream flash attention to support Navi GPUs before targeting the consumer architectures during compilation. If you'd like to help out with testing that and/or helping with development, you can send me a DM on discord: alpindale