Online executor for large scale datasets

[thekej]

• The current brainscore implementation is optimized to handle small datasets to study neural alignment or behavior on small datasets. While this is great, it does not give user the ability to test their model activations on larger scale datasets.
• We have tested the practicality of using brainscore for getting metrics on larger scale datasets (7k videos) but we have ended up with the following challenges:

1. No behavioral readout adapted for videos, or more generally any data that depends on time. Current pipeline only supports logistic regression readouts.
2. Inability to do data augmentation during the training of the readout. According to our experiments, data augmentation is essential in this readout task.
3. Extraction is handled such that everything is put on ram during execution. while this is fine for image models with limited datasets, it becomes a huge limitation when you are working with large scale datasets.

What we propose in this PR:

1. A transformer based readout mechanism added as a new task in behavior. Supports usual training mechanism as well as online training.
2. An online executor that handles, data augmentation, feature extractions and readout training.
3. A new inferencer is added to use this executor.

This implementation does not affect any original functionality of brainscore.

[mike-ferguson]

@thekej wow - this is great! Thanks so much for opening a PR; our testing suite is being ported to AWS currently, so things might be a bit rocky while we sort this out. @mschrimpf I am tagging you for review here, as support for video has been a long standing goal of Brain-Score.

[mschrimpf]

@YingtianDt

[YingtianDt]

In brainscore_vision/model_helpers/activations/temporal/core/executor.py, a readout module is directly trained within the executor so that it gets trained online. The executor then returns "dummy_activations", which basically breaks the APIs currently used by the downstream modules.

This is an interesting construct, but I encourage the author to use it locally at this moment. We can talk in the future to see if there is a better way to incorporate these ideas.