About
With rapid advances in large language models (LLMs), there has been an increasing application of LLMs in creative content ideation and generation. A critical question emerges: can current LLMs provide ideas that are diverse enough to truly bolster collective creativity?
We examine two state-of-the-art LLMs, GPT-4 and LLaMA-3, on story generation and discover that LLM-generated stories often consist of plot elements that are echoed across a number of generations. To quantify this phenomenon, we introduce the Sui Generis (SG) score, an automatic metric that measures the uniqueness of a plot element among alternative storylines generated using the same prompt under an LLM. The higher the score is, the more unique the plot is.
Example
The blue segments are the more unique ones with high SG scores, while the red ones are the low-scored segments.
Click on each segment to check its Sui Generis score and more details.