CREATE EXTENSION plproxy, and it will transparently download and build plproxy for you. (Actually, this only works if the extension name is the same as the package name. I'm planning to fix that.)
Note 1: You can't install Autopex via Pex, yet.
Note 2: I guess the next logical step would be Autoautopex, which installs Autopex and Pex automatically somehow. Patches welcome.
I suppose with logical replication, this might actually end up installing the extension code on the replication slaves as well. That would be pretty neat.