Owain Evans on X: "We got the teacher model to generate a dataset of word sequences and Alpaca responses, filtering out digits and any words related to the MLP task. A fresh student was finetuned on this and it learned to predict the MLP (yet still weaker than the teacher)! https://t.co/1OKteTDM7L" / X
x.com · saved by 1 readers
We got the teacher model to generate a dataset of word sequences and Alpaca responses, filtering out digits and any words related to the MLP task. A fresh student was finetuned on this and it learned to predict the MLP (yet still weaker than the teacher)!