%20tasks%2C%20the%20demand%20for%20these%20state-of-the-art%20models%20increases%20commensurately.%20However%2C%20their%20deployment%20is%20costly%20both%20in%20terms%20of%20processing%20power%20and%20time.%20In%20this%20paper%2C%20we%20investigate%20the%20feasibility%20of%20creating%20smaller%2C%20highly-performing%20specialized%20algorithms%20by%20way%20of%20fine-tuning.%20To%20do%20this%2C%20we%20first%20present%20a%20large%20pre-trained%20model%20with%2020%20unique%20scenarios%20that%20combine%20different%20social%20contexts%20with%20games%20of%20varying%20social%20dilemmas%2C%20record%20its%20answers%2C%20and%20use%20them%20for%20Q%26A%20fine-tuning%20on%20a%20smaller%20model%20of%20the%20same%20family.%20Our%20focus%20is%20on%20in-context%20game-theoretic%20decision-making%2C%20the%20same%20domain%20within%20which%20human%20interaction%20occurs%20and%20that%20requires%20both%20a%20theory%20of%20mind%20(or%20a%20semblance%20thereof)%20and%20an%20understanding%20of%20social%20dynamics.%20The%20smaller%20model%20is%20therefore%20trained%20not%20just%20on%20the%20answers%20provided%2C%20but%20also%20on%20the%20motivations%20provided%20by%20the%20larger%20model%2C%20which%20should%20contain%20advice%20and%20guidelines%20to%20navigate%20both%20strategic%20dilemmas%20and%20social%20cues.%20We%20find%20that%20the%20fine-tuned%20smaller%20language%20model%20consistently%20bridged%20the%20gap%20in%20performance%20between%20the%20smaller%20pre-trained%20version%20of%20the%20model%20and%20its%20larger%20relative%20and%20that%20its%20improvements%20extended%20in%20areas%20and%20contexts%20beyond%20the%20ones%20provided%20in%20the%20training%20examples%2C%20including%20on%20out-of-sample%20scenarios%20that%20include%20completely%20different%20game%20structures.%20On%20average%20for%20all%20games%2C%20through%20fine-tuning%2C%20the%20smaller%20model%20showed%20a%2046%25%20improvement%20measured%20as%20alignment%20towards%20the%20behavior%20of%20the%20larger%20model%2C%20with%20100%25%20representing%20indistinguishable%20behavior.%20When%20presented%20with%20out-of-sample%20social%20contexts%20and%20games%2C%20the%20fine-tuned%20model%20still%20displays%20remarkable%20levels%20of%20alignment%2C%20reaching%20an%20improvement%20of%2018%25%20and%2028%25%20respectively. "Large Model Strategic Thinking, Small Model Efficiency: Transferring Theory of Mind in Large Language Models")
Large Model Strategic Thinking, Small Model Efficiency: Transferring Theory of Mind in Large Language Models
Original caption: Figure 1: Overview of the methods employed in this paper. We pair all games and scenarios to generate 20 unique combinations, which form the backbone of our dataset. We then submit each combination to each model, and obtain 300 observations per combination. For LLaMa2-70b, we ask for an answer and a motivation; we ask the other models only for their answers. We use the answers coming from LLaMa2-70b to perform LORA on a small, pre-trained LLaMa2-7b. The fine-tuned model is then again queried like the pre-trained model, and once that is done, we collect all data and measure the impact of fine-tuning on preferences.
Original caption: Figure 2: Preliminary investigation of differences in responses between LLaMa2-7b and LLaMa2-70b, grouped by context and game. Clockwise, from the left: propensity to cooperate in LLaMa2-7b grouped by context; propensity to cooperate in LLaMa2-7b grouped by game; propensity to cooperate in LLaMa2-70b grouped by context; propensity to cooperate in LLaMa2-70b grouped by games. Notably, LLaMa2-7b is almost entirely indifferent to context and game and displays a remarkable bias for choosing cooperation, whereas LLaMa2-70b adapts to new contexts and game structures to a remarkable extent.
Original caption: Table 1: Difference in proportion z𝑧zitalic_z-score testing for propensity to cooperate in within-sample and out-of-sample games for the LLaMa2-7b fine-tuned model. For each scenario, we report: the proportion of cooperative choices in the within-sample game, the proportion of cooperative choices in the out-of-sample game, the difference in proportions, standard error, z𝑧zitalic_z-score, and associated p𝑝pitalic_p-value. Reported significance levels follow standard practices: one asterisk (*) for significance at the 0.050.050.050.05 level, two asterisks (**) for significance at the 0.010.010.010.01 level, and three asterisks (***) for significance at the 0.0010.0010.0010.001 level.
/cdn.vox-cdn.com/uploads/chorus_asset/file/25290355/STK048_XBOX_A.jpg)
