The increasing volume of data generated by AI threatens the future reliability of Big Language Models.
By 2028, 50% of organizations will implement a posture of zero-trust for data governance due to the proliferation of unverified data generated by Artificial Intelligence (AI), according to Gartner, Inc.., a business insights and technology company.
“"Organizations can no longer implicitly trust data or assume that it was generated by humans," he says. Wan Fui Chan, Vice President of Management at Gartner. “As AI-generated data becomes ubiquitous and indistinguishable from human-created data, a stance zero-trust "Establishing authentication and verification measures is essential to protect business and financial outcomes."”
Data generated by AI increases the risks of model collapse and compliance requirements.
Large Language Models (LLMs) are typically trained with data "collected from the web" and a variety of sources, including books, code repositories, and research articles. Some of these sources already contain AI-generated content, and if current trends continue, almost all will eventually be populated with AI-generated data.
According to the 2026 Gartner CIO and Technology Executive Survey, 84% of respondents expect their organizations to increase funding for Generative Artificial Intelligence (GenAI) by 2026. As organizations accelerate the adoption and investment in AI initiatives, the volume of data generated by Artificial Intelligence will continue to increase. This means that future generations of LLMs will be increasingly trained based on the results of previous models, increasing the risk of “model collapse,” where the responses of AI tools may no longer accurately reflect reality.
“As AI-generated content becomes more prevalent, regulatory requirements to verify 'AI-free' data are expected to intensify in certain regions,‘ says Chan. ’However, these requirements may differ significantly across geographic regions, with some jurisdictions seeking to apply stricter controls to AI-generated content, while others may adopt a more flexible approach.”.
“In this evolving regulatory environment, all organizations will need to have the ability to identify and mark (tag) in data generated by AI. Success will depend on having the right tools and a skilled workforce in information and knowledge management, as well as metadata management solutions that are essential for data cataloging.”.
Active metadata management practices will become a key differentiator, enabling organizations to analyze, alert on, and automate decision-making across all their data assets.
Managing risks associated with the rise of unverified data generated by AI.
Organizations should consider several strategic actions to manage the risks of unverified data:
– Appoint an AI governance leader: Establish a position responsible for Artificial Intelligence governance, including policies for... zero-trust, AI risk management and compliance operations. This leader must work closely with the Data & Analytics (D&A) teams to ensure that data is... Ready for Artificial Intelligence and that the systems are capable of handling AI-generated content.
– Promote cross-functional collaboration: form cross-functional teams that include cyber security, Data & Analytics and other relevant stakeholders to conduct comprehensive data risk assessments, seeking to identify business risks related to AI-generated information and determine which are addressed by existing data security policies and which require new strategies.
– Leverage existing governance policies: base yourself on governance structures Focus on updating current Data & Analytics policies and concentrate on updating policies related to security, metadata management, and ethics to address new data risks generated by AI.
– Adopt active metadata practices: This allows for real-time alerts when data is outdated or needs recertification, helping organizations quickly identify when business-critical systems may be exposed to inaccurate or biased information.
Topics such as this and others related to data, security, and risk management will be addressed during the event. Gartner Data & Analytics Conference, held on April 28 and 29, and Gartner Conference Security & Risk Management, which takes place on August 4th and 5th in São Paulo.
Gartner clients can learn more at “Predicts 2026: AI-Driven, Automated, Zero-Trust Governance”Learn how to maximize the value of AI while managing critical risks with the free e-book “AI in Cybersecurity: Minimize Risks and Maximize Impact”.
Gartner is the world authority on AI.
Gartner is the indispensable partner for C-level executives and technology vendors as they implement AI strategies to achieve their mission-critical priorities. The independence and objectivity of Gartner's insights provide clients with the confidence needed to make well-informed decisions and unlock the full potential of AI. C-level clients are using Gartner's AI tool. AskGartner to define how to leverage AI in your business. With over 2,500 business and technology experts, 6,000 written insights, plus more than 1,000 AI use cases and case studies, Gartner is the world authority on AI. More information can be found [here/at this link]. on here.
About the Gartner Security & Risk Management Conference
Gartner analysts will present the latest insights for security and risk management leaders during the Gartner Security & Risk Management Conferences, which will be held on March 9th and 10th in [location missing from original text]. Mumbai (India), March 16 and 17 Sydney (Australia), June 1-3 National Harbor (USA), July 22-24 in Tokyo (Japan), August 4 and 5 Sao Paulo (Brazil) and September 22-24 in London (United Kingdom). Follow the news and updates from the conferences at X and LinkedIn using #GartnerSEC.
About the Conference Gartner Data & Analytics
Gartner analysts will provide further analysis on Data and Analytics trends at the Gartner Data & Analytics Conferences, which will take place from March 9-11 in [location missing from original text]. Orlando (United States), April 28 and 29 Sao Paulo (Brazil), May 11-13 London (United Kingdom), May 19-21 Tokyo (Japan), June 16 and 17 in Sydney (Australia) and September 21 and 22 in Mumbai (India). Follow the news and updates from the conference at X and LinkedIn using #GartnerDA.
About Gartner for Data & Analytics Leaders
The Gartner for Data & Analytics Leaders provides objective and actionable insights for CDAOs and Data & Analytics leaders, helping them accelerate their D&A strategy and operating model to increase business value. Additional information is available at [link to Gartner for Data & Analytics Leaders]. https://www.gartner.com/en/data-analytics. Follow the news and updates from Gartner for Data & Analytics Leaders atXandLinkedIn using #GartnerDA.
About Gartner
O Gartner, Inc. delivers objective, actionable insights that drive smarter decisions and better performance for enterprises’ mission-critical priorities. To learn more, visit www.gartner.com.
