skip to content
Nikolay Donets

Self-Generated In-Context Learning Explained

/ 2 min read

Traditional in-context learning for LLMs relies on carefully curated examples, but what if the model could generate its own demonstrations? An idea called Self-Generated In-Context Learning SG-ICL explores this possibility.

How SG-ICL solves the problem of traditional LLM task setting

Typically, when we use LLMs, we rely on a process called in-context learning. We feed the model examples to demonstrate the task we want it to do. These examples are critical – bad ones mean poor results. It can be a lot of work to find the right data, and you never know if what you choose will ultimately lead to the best outcome.

Instead of looking for examples, SG-ICL prompts the LLM to create its own. Here’s how it works:

  1. Guided generation. We give the LLM a bit of context. We provide the specific task instructions and the kind of output we expect, basically telling it what it should write.
  2. LLM writes the examples. LLM generates text samples that fit the task.
  3. Learn from within. Self-generated examples go straight back into the context, removing the need for any external data.


  • Improved performance. Get better results without having to find or create additional training data.
  • Greater consistency. As the LLM creates the examples, performance becomes more consistent.

Could SG-ICL work for you?

This approach has great potential:

  • Custom solutions. Adapting language models for specific tasks.
  • Solve data scarcity issues. Particularly helpful when a large data set is not readily available.