Synthetic Data Tools Struggle to Keep Pace With AI-Driven Development Demands

Artificial intelligence has fundamentally reshaped how software engineers work. Adoption of AI-powered coding assistants has roughly doubled year-over-year, and more than 90% of professional developers now rely on AI within their everyday tasks. Testing represents one of the most promising applications for AI agents in development: they can compose, modify and execute test suites far faster than human engineers, while simultaneously exploring novel test scenarios and uncovering edge cases at scale.

Yet this rapid shift has created a significant market problem. As AI agents handle more testing responsibilities, the volume and nature of required test data has transformed dramatically—and most vendors supplying synthetic data solutions have failed to adapt. A survey of 518 enterprise technology leaders conducted on behalf of Perforce Delphix revealed a puzzling gap: two-thirds of respondents (66%) reported that their organizations do not employ synthetic data for development and testing. Many cited concerns that available solutions lack the realism and data consistency their enterprises demand.

Agentic development workflows demand test data in greater quantities and at faster delivery speeds than traditional manual development ever required. When operating at enterprise scale, these systems also need test data that maintains consistency across interconnected systems and preserves the statistical characteristics of real-world information. Until recently, synthetic data vendors have offered products that excelled at one of these requirements or the other—but not both. Perforce Delphix's newly released Synthetic Data platform claims to be the first solution delivering both properties simultaneously.

Capable, efficient and data-hungry: AI's role in software development

AI assistance can accelerate software development by up to 2x, provided the AI agents remain continuously engaged. With enterprise deployment of coding agents accelerating and pricing models shifting toward consumption-based structures, idle time for AI agents has become economically unsustainable. To prevent agents from sitting unused, development teams frequently resort to low-quality or outdated test data, or worse—they pull unprotected production data into testing environments.

We have long heard complaints about how long it took for test data to become available. Especially if it was being copied from production. But today, with the rise of agentic development, the costs are higher than ever. Enterprises simply can't afford to have their agents wait for production data.

Mayank Ahluwalia, senior product manager for Delphix Synthetic Data and data compliance for AI at Perforce Delphix

Synthetic data addresses this challenge because it can be provisioned nearly instantaneously. It also scales efficiently, which matters when workflows involve numerous agents operating concurrently. Additionally, AI-driven development requires test data with substantial variety.

When agents are writing test cases, they're not constrained by the limits of human imagination. As humans, we can only test the scenarios we can think of, and our ideas are shaped by our biases. Agents use combinatorial logic to come up with a much wider variety of novel scenarios. Teams need test data that's suitable for all of them.

Ahluwalia

Yet agentic development also requires test data that faithfully mirrors real-world conditions. To accomplish this, synthetic data must possess two essential attributes:

• Referential integrity: Data maintains referential integrity when all components remain correctly linked across systems. In an e-commerce setting, every order record should correspond to a valid customer account. A human tester might catch a missing customer link, but AI agents will continue testing and may incorrectly mark such cases as passing.
• Statistical realism: When data trains models, it must exhibit statistical properties matching production data; otherwise, the model absorbs incorrect patterns. Salary information, for instance, should display the same distribution, floor and ceiling values as actual data.

Historically, development and testing tools have typically produced data with some degree of referential integrity, though only after substantial configuration work, while model training tools have prioritized statistical realism. Among survey respondents who had tested existing synthetic data products, just 36% reported that the data demonstrated statistical realism. Even fewer—34%—confirmed that their synthetic data possessed referential integrity.

Born-in-AI solution overcomes the bottleneck

Perforce Delphix has launched a new Synthetic Data platform engineered specifically to address this market gap. The solution employs multiple AI methodologies—including natural language processing, machine learning, large language models and agent capabilities—to automatically map data structures, interconnections and organizational context across data sources, eliminating the manual setup that older synthetic data tools require. Using a prompt-driven interface, developers and AI agents can produce realistic, scenario-tailored data within seconds.

Our solution looks at the schema to understand what information each field holds and what that information means. It uses this understanding to generate data with referential integrity, and it samples the data to learn its statistical distribution. The included chat assistant is not just bolted on as it would be in a legacy solution. Instead, it can actively guide you by making recommendations about how to best generate data for the types of scenarios you need to test.

Ahluwalia

While the governance and compliance advantages of synthetic data are substantial, the rationale for adoption extends well beyond regulatory requirements.

Historically, people have considered synthetic data a compliance solution. When you look at it that way, you leave a lot of the benefits on the table. If you can create synthetic data to expedite agentic development and testing, you'll start reaping the full benefits of your investments in AI.

Ahluwalia