Large language model compliance with content marketing rules

Abstract

This report identifies a need for discovering the best performing method for aligning LLMs to a set of rules so that brands can use the aligned LLMs to generate marketing content with ease of mind that all of the brand’s content marketing rules will be adhered to. The report compares Direct Preference Optimisation (DPO), the new best performing preference learning method, and a newly created method for this report named Generator-Validator. The method is based on human psychology, applying the methods in which a human would best learn a set of rules to an LLM. Evaluation finds the Generator-Validator method to be a more effective and efficient method than DPO, with error rates varying between 1.00% and 20.99% for the four sets of rules tested. This method also shows significant improvement in rule compliance compared to passing the rules to the model within the prompt, with error rates improving up to 66.01%. This report is focused on generating the methods for L’Oreal Paris. However, all methods are tested sufficiently enough that they could be used with any company’s content marketing rules.