The Library of Tasks

Large Language Models and Borges Library of Babel

The library of Babel created using DallE OpenAI

"The Library of Babel" is a short story written by the Argentine author Jorge Luis Borges, which was first published in 1941 as part of his collection titled "Ficciones." The story is a philosophical and metaphysical exploration of themes such as infinity, knowledge, and the nature of reality. It is a good analogy of the data input and output from Large Language Models (LLMs) such as Google Bard, GPT4, Claude, etc..

The plot revolves around a seemingly endless library. This library is made up of an enormous number of interconnected hexagonal rooms, each containing books that are all exactly the same in terms of format and content. The library contains every possible combination of letters and spaces, meaning that it contains every book that has ever been written, every book that could be written, and a vast number of nonsensical or meaningless books.

The librarians, who inhabit this world, are engaged in a seemingly futile quest to find some order or meaning within the vast chaos of books. They hope to discover a "Vindications," a book that would provide a key to understanding the library's contents. The protagonist of the story is one such librarian who becomes obsessed with the search for meaning within the library. Just like the best prompt engineers seeking to maximize the output accuracy from AI.

As the story progresses, it delves into the idea that the library represents the universe itself, with all human knowledge contained within it, both the meaningful and the meaningless. The search for meaning and order within the library is a metaphor for the human quest for understanding and purpose in a seemingly chaotic and infinite world.

Borges' story raises profound questions about the limitations of human knowledge, the quest for meaning in a vast and potentially meaningless universe, and the paradoxes of infinite information. "The Library of Babel" is a thought-provoking and often surreal exploration of these themes and has become a classic of literature, known for its intellectual depth and imaginative concepts.

The story of the Library of Babel illustrates how the infinite possibility of text can be overwhelming and disorienting. Similarly, the output of a Large Language Model can be difficult to make sense of. But with careful curation and filtering, it can also produce insightful and highly creative text.

In the near future I think we will move from the power of text creation with AI (including coding) to the power of task completion - the number of tasks AI can currently complete is mind boggling and we have hardly scratched the surface.