[Library Forwarding/Question] Any plan to forward openssl?

[ChanthMiao]

Is there any plan to support library forwarding for openssl(libcrypto.so and libssl.so)?

[Sonicadvance1]

Usually we only create library forwards for things that take significant CPU time in a wide variety of applications like GL and Vulkan. Each one adds maintenance burden. Is there some application spending significant time in this library to matter?

[ChanthMiao]

First of all, I would like to thank the FEX team for their efforts.

I have successfully ported FEX to a DEC/ALPHA like architecture For the most part, it performs well (classic games like Plants vs. Zombies run well). However, the architecture does not provide support for encryption, decryption, and hash computation-related instructions, so I can only simulate instructions using the helper function. A large number of Spill/Fills combined with poor memory read/write performance have led to inefficient encryption and decryption translation.

When the app involves intensive web resource pulling (such as the Steam homepage display and WeChat's built-in browser), it performs poorly in my local private environment. Of course, this is not FEX's fault. On the FT-D3000 (armv8), there are absolutely no such performance issues.

I originally planned to implement openSSL forwarding support myself, but after an initial attempt, I was temporarily stumped by the callback parameters and variadic parameters of libcrypto and libssl. I tried to check the relevant historical information and saw the term libssl in #267. So regarding the question mentioned in #267, are there any related plans for libssl?

[ChanthMiao]

Partly referencing the implementation of box64, I have solved the forwarding problem of variadic parameter functions by declaring custom function to forward and refactoring va_list structures. However, I have not yet found a solution for the complex callback parameters.

[ChanthMiao]

Here is my implement details.

For variadic parameter functions, e.g. int BIO_printf(BIO* bio, const char* format, ...) and int BIO_vprintf(BIO* bio, const char* format, va_list sysv_vaargs):

1. Declaring helper function int fex_internal_BIO_printf(BIO *bio, const char *format, void *) in shared headers.
2. In interface.cpp, forward th helper function with custom guest and host impl.

template<typename>
struct fex_gen_type {};

template<auto, int, typename = void>
struct fex_gen_param {};

template<>
struct fex_gen_type<BIO> : fexgen::opaque_type {};

template<>
struct fex_gen_config<fex_internal_BIO_printf> : fexgen::custom_host_impl, fexgen::custom_guest_entrypoint {};

3. In Guest.cpp, pack variadic parameters, and pass x64 va_list pointer as void *

extern "C" {
int BIO_printf(BIO* bio, const char* format, ...) {
  va_list ap;
  va_start(ap, format);
  int result = fexfn_pack_fex_internal_BIO_printf(bio, format, &ap);
  va_end(ap);
  return result;
}

int BIO_vprintf(BIO* bio, const char* format, va_list sysv_vaargs) {
  return fexfn_pack_fex_internal_BIO_printf(bio, format, &sysv_vaargs);
}
}

4. In Host.cpp, build aarch64 host va_list from x64 va_list.

static int FEXFN_IMPL(fex_internal_BIO_printf)(BIO* bio, const char* format, void* sysv_vaargs) {

  auto x64_vaargs = reinterpret_cast<x64_va_list_s*>(sysv_vaargs);

  va_list host_ap;
  host_ap.__gr_offs = x64_vaargs->gp_offset - (6 * 8);
  host_ap.__vr_offs = x64_vaargs->fp_offset - ((6 * 8) + (8 * 16));
  host_ap.__stack = x64_vaargs->overflow_arg_area;
  host_ap.__gr_top = reinterpret_cast<char*>(x64_vaargs->reg_save_area) + (6 * 8);
  host_ap.__vr_top = reinterpret_cast<char*>(x64_vaargs->reg_save_area) + ((6 * 8) + (8 * 16));

  if (LDR_PTR(BIO_vprintf) == nullptr) {
    (void*&)LDR_PTR(BIO_vprintf) = dlsym_default(fexldr_ptr_libcrypto3_so, "BIO_vprintf");
  }

  return LDR_PTR(BIO_vprintf)(bio, format, host_ap);
}

For functions with multi function pointers(Single-direction setup, later only used internally; the guest side will need to read function pointers through other external interfaces going forward). Inspired by libwayland-client, pass thoes pointers as void **.

1. Declaring helper function int fex_internal_CRYPTO_get_ex_new_index(int class_index, long argl, void* argp, void (**callback)(void)) in shared headers.
2. In interface.cpp, forward th helper function with custom guest and host impl.
3. In Guest.cpp, pack function ponter trampoline as arrary.

extern "C" {
int CRYPTO_get_ex_new_index(int a_0, long a_1, void* a_2, CRYPTO_EX_new* new_func, CRYPTO_EX_dup* dup_func, CRYPTO_EX_free* free_func) {
  uint64_t host_callbacks[3];
  host_callbacks[0] = CryptoAllocateHostTrampolineForGuestFunc(new_func);
  host_callbacks[1] = CryptoAllocateHostTrampolineForGuestFunc(dup_func);
  host_callbacks[2] = CryptoAllocateHostTrampolineForGuestFunc(free_func);

  return fexfn_pack_fex_internal_CRYPTO_get_ex_new_index(a_0, a_1, a_2, (void (**)())host_callbacks);
}
}

4. In Host.cpp, extract trampoline.

static int FEXFN_IMPL(fex_internal_CRYPTO_get_ex_new_index)(int a_0, long a_1, void* a_2, guest_layout<void (**)(void)> a_3) {
  // Replace guest-provided callback table with host-callable function pointers
  auto callbacks = a_3.force_get_host_pointer();

  auto new_func = (CRYPTO_EX_new*)callbacks[0];
  auto dup_func = (CRYPTO_EX_dup*)callbacks[1];
  auto free_func = (CRYPTO_EX_free*)callbacks[2];

  FinalizeHostTrampolineForGuestFunction(new_func);
  FinalizeHostTrampolineForGuestFunction(dup_func);
  FinalizeHostTrampolineForGuestFunction(free_func);

  if (LDR_PTR(CRYPTO_get_ex_new_index) == nullptr) {
    (void*&)LDR_PTR(CRYPTO_get_ex_new_index) = dlsym_default(fexldr_ptr_libcrypto3_so, "CRYPTO_get_ex_new_index");
  }

  return LDR_PTR(CRYPTO_get_ex_new_index)(a_0, a_1, a_2, new_func, dup_func, free_func);
}

But, for function pointers that need to be read later, e.g. void BIO_set_callback_ex(BIO *b, BIO_callback_fn_ex callback) and BIO_callback_fn_ex BIO_get_callback_ex(const BIO *b), I do not know how to handle them. Unlike libvulkan and libwayland-client, the callback paramter here is free to be any user provided function.

@Sonicadvance1 Do you know any method base in current code base to handle the lasted case? I can do the rest tedious work by myself.

[neobrain]

But, for function pointers that need to be read later, e.g. void BIO_set_callback_ex(BIO *b, BIO_callback_fn_ex callback) and BIO_callback_fn_ex BIO_get_callback_ex(const BIO *b), I do not know how to handle them. Unlike libvulkan and libwayland-client, the callback paramter here is free to be any user provided function.

For BIO_set_callback_ex, Why can't you use CryptoAllocateHostTrampolineForGuestFunc(callback) and pass the result to LDR_PTR(BIO_set_callback_ex)?

BIO_get_callback_ex is a bit trickier, but since we know what the host trampolines look like, a GetGuestFromHostTrampoline function could probably be added to Thunks.cpp to make this work. That's assuming the callback is only ever set through these APIs, if that's not true we need something else.

Would definitely be interested in seeing if this improves performance for the use case you mentioned, then we can look into including it in the main repo as well.

[ChanthMiao]

For BIO_set_callback_ex, Why can't you use CryptoAllocateHostTrampolineForGuestFunc(callback) and pass the result to LDR_PTR(BIO_set_callback_ex)?

Yes, the main problem is that, BIO_get_callback_ex may return the pointer set by BIO_set_callback_ex

BIO_get_callback_ex is a bit trickier, but since we know what the host trampolines look like, a GetGuestFromHostTrampoline function could probably be added to Thunks.cpp to make this work. That's assuming the callback is only ever set through these APIs, if that's not true we need something else.

Thanks, I will try it. This might take quite a bit of my time, as I've been quite busy with work lately.