Introduction

The Key Technologies of Knowledge-enhanced Large Language Model enable integrated learning from trillions of data and hundreds of billions of knowledge. The breakthroughs in knowledge internalization and externalization bring better model performance and efficiency, thanks to unique advantages of knowledge enhancement, retrieval enhancement and dialogue enhancement, as well as the joint optimization with the PaddlePaddle platform.

Innovative Technologies of Knowledge-enhanced Large Language Models

Knowledge enhancement: a breakthrough in knowledge internalization and externalization that solves the difficulties of representing and utilizing data and knowledge in a unified manner, and helps the model winning the first place at SuperGLUE with 0.8 percentage points beyond human level.

Retrieval enhancement: the joint optimization of the search engine and the large language model has increased the accuracy and timeliness of the generated content. 

Dialogue enhancement: with the help of the world’s first hundred-billion-parameter dialogue large model which has won 11 champions at the Dialogue System Technology Challenge, the large language model has obtained the capabilities of dialogue memory, in-context understanding and dialogue planning, and is able to produce more coherent and reasonable dialogues.

Knowledge point enhancement: the knowledge point enhancement in both input and output stages has greatly increased the model's efficiency and performance.

Agent Mechanism: The proposed agent mechanism includes the abilities of understanding, planning, reflecting and evolving, resulting in an intelligent agent that is capable of reliable execution, self-evolvement and shows an interpretable thinking process.

Joint Optimization of Model and Deep Learning Framework：The joint optimization of the large language model ERNIE and the deep learning platform PaddlePaddle makes it possible to iterate rapidly. As a result, the inference efficiency of the model has increased over 50 times.

Intensive Production for Large Models and Platform Based Empowerment for Industries

The key technologies have been applied in Intelligent Search, General-purpose Dialogue and ERNIE large models, etc., and have empowered a large number of industries through PaddlePaddle, the open-source deep learning platform by lowering the threshold for artificial intelligence technology innovation and application, and accelerating industrial transformation and upgrading.

Intelligent Search powered by technologies of knowledge-enhanced large language model delivers more accurate results and better search experience to satisfy billions of users’ demands. The technologies also make the development more efficient and the innovation process faster.

General Dialogue system powered by these technologies has applied in more than 20 industries such as telecommunication, energy, finance and media, etc. and served over a billion users on over 500 million smart devices, bringing economic benefit of nearly 8 billion Yuan.

ERNIE knowledge-enhanced large models, including foundation model and industrial model, enable customers and users to easily develop and deploy the entire process of AI applications through toolkit and platform. More than 150,000 customers and partners have applied to access the big language model platform, successfully practicing the large model industrial model of "intensive production and platform application", and empowering industrial intelligent upgrading.

Reshape the Industry, Transform Research Paradigm and Support the exploration of AGI

The breakthrough of key technologies of knowledge-enhanced large language model would help explore the way to AGI, transform research paradigm to speed up big inventions and discoveries, and reshape the industry to boost the wave of industrial upgrading and transformation.

The knowledge-enhanced large language model is capable of understanding, generating, logic and memorizing known as AI's fundamental abilities, which provides a technical foundation for the development of AGI.

The knowledge-enhanced large language model performs well in interdisciplinary or cross-disciplinary tasks, which helps scientists to integrate academic resources, experiment with new ideas and ideas, and produce new breakthroughs and discoveries faster.

The knowledge-enhanced large language model would foster new industries or new business opportunities such as cloudAI integrated model as a service, new market boosted by industrial large models and new phenomenal applications. Reportedly, AI technologies, mainly large language models and generative AI, could contribute up to $ 15.7 trillion to global economy in 2030.