Make this work with LibTorch 2.12

[slburson]

This now successfully runs on LibTorch 2.12, producing a bindings file that compiles. I haven't built the adapter yet. Besides fixing some Claw bugs, I did the following:

• arranged to use a binary LibTorch distribution, so you don't have to build PyTorch from source
• created a patch for one LibTorch header file that was tickling a couple of Claw bugs, to work around them
• tweaked the defwrapper form to include names in torch::nn::
• added directions to the README

I'm opening this PR so you can see what I've been doing, but it isn't ready to merge yet. In fact — what's your feeling about this project? Do you want to keep control of it, or are you happy to let me take it over? If the latter, we can just consider my fork to be the official repo. Do you mind if I rename it to CL-Torch?

[slburson]

Ha ha, the joke was on me: getting the bindings file to compile was the easy part; the adapter was much harder. For starters, I had to make some changes to its CMakeLists.txt, to force it to build with Clang and the Clang headers only.

The big problem was with inherited constructors. Example:

struct A {
    A(int);
    A(char *);
};

struct B : A {
    using A::A;
};

Here B inherits all the declared constructors of A; also, neither class has a default constructor. Unfortunately, libclang does not put the correct constructor set on the CXType (I have filed a bug). It is possible to find the using declaration, with the set of constructors to be inherited, by walking the decl for B; I spent quite some time working out how to do this, but eventually I decided to table that work and just edit out the construct from the LibTorch headers. This wasn't completely trivial either, but I managed to do it. (See libtorch-patch-01-inherited-constructors in this repo.)

Once that was done, there were just a few minor bugs in the adapter generation that I had to fix, and it built! 🚀

Then I had to get the shared library to load. That wasn't quite trivial either. The first problem was that Claw uses the Clang headers if available (see claw.util::list-system-paths), but LibTorch was compiled against the GCC headers. This showed up as different mangled names for functions with a parameter of type ostream&. (There could well have been other problems; dlopen gives up after the first error.) I temporarily hacked list-system-paths to ignore Clang, and tried a couple of different GCC header tree versions; 15 didn't work, but 10 did.

After that:

• The two static methods in torch::nn::functions::CrossMapLRN2d (declared in torch/csrc/api/include/torch/nn/modules/_functions.h) were evidently compiled with visibility=hidden, because they were private to the LibTorch library.
• Function torch::getAllCustomClassesNames (declared in torch/custom_classes.h) was also hidden. In this case, I think the hiding may be inadvertent.
• at::TensorIteratorBase::apply_perm_and_mul (declared in ATen/TensorIterator.h) is simply unimplemented.

For the moment, I have hand-edited these out of the adapter and bindings files. I guess I'll patch the headers again to get rid of them.

And with that — it loads!!! 🚀 🚀 🚀

Whew! Now let me try calling some functions...

[slburson]

I have extended Claw to handle inherited constructors automatically. So the patch (libtorch-patch-01-inherited-constructors) that edits them out of the LibTorch headers is no longer needed.

I think this is now done, or nearly so. The only thing remaining, AFAIK, is to prune the set of names for which bindings are generated; the bindings file is ~5MB and the adapter ~3.7MB, and it's reasonable to assume there's stuff in here we won't actually need, but I'm not inclined to worry about that now. Of course, it's also possible we'll find names that need to be added to the binding set.

@borodust I am very grateful to you for your work on Claw, which made this progress possible. I did have to make some improvements; LibTorch is a large and complex library that uses some of the gnarlier features of C++ — it seems like a pretty stringent test case for Claw. I think I've contributed significant value in getting it working, and in preparing clean PRs for your perusal.

So I ask again: what is your feeling about Claw-Torch? There's no license file, and it does contain some hand-written code, though not very much. So I feel as if I should ask before doing what I want with it; but if you don't speak up soon, I will take your silence as permission. To repeat, what I want to do is to make it part of a larger project called CL-Torch, with an API layer written in CL. Never mind: I see that the defsystem forms all have :license "MIT". Great!