Augmenting LLMs Beyond Basic Text Completion and Transformation

Large Language Models (LLMs) have shown remarkable capabilities in various natural language processing tasks such as summarization, translation, and generation. Recent advancements in LLMs have led to emergent abilities like text-to-code conversion, which has been utilized by products like Github's Copilot. As these models continue to scale in complexity and robustness, they are expected to yield more "second-order" applications beyond core NLP tasks. However, LLMs still suffer from hallucinations or non-factual responses due to erroneous encoding and decoding by the transformer or divergences in training data. To tackle these issues, researchers propose augmenting LLMs with external tools like calling another fine-tuned model, retrieving information via search engine or internet, solving computational problems via code interpreter or calculator, etc. This approach not only minimizes errors and hallucinations but also grants LLMs capabilities outside of textual generation. Several "second-order" applications have already been developed, such as intelligent software agents like Siri and Alexa, supercharged search engines, and hardware interfaces for robots. These applications leverage the model's response to perform another action or sequence of actions. As LLMs continue to evolve, it is possible that they may lead to even more abstract, complex "third-order" applications in the future, potentially bringing us closer to artificial general intelligence (AGI).