Introduction

This code is the implement of our XXX 2020 paper (under review). Our model is based on mt-dnn. The main difference is the sampling and training strategy used in this paper, which is the file ./mt_dnn/acl_controller.py.

Preparation

The glue experiments are defined in experiments/glue, while the transfer learning experiments are defined in experiments/transfer

Environment setups:

pip install -r requirements.txt

Data Preprocessing

Download GLUE data

sh download.sh

Please refer to download GLUE dataset: https://gluebenchmark.com/

Preprocess Glue data

sh experiments/glue/prepro.sh

Download transfer learning data

TODO

Preprocess data

sh experiments/transfer/prepro.sh

Train Model

Train model

sh scripts/acl_controller.sh 0.5

where the first argument is the \phi value in our policy

Transfer Learning

TODO

Citation

TODO