AI-driven Personalized Pricing Under Scrutiny by States

1 week ago 0

Imagine two individuals purchasing the same product online simultaneously, yet encountering different prices. This scenario is at the core of mounting concerns about surveillance pricing and AI-driven personalized pricing systems. These issues have prompted Maryland to become the first state, followed by Connecticut, to implement laws limiting certain forms of surveillance pricing. New York and New Jersey have also passed similar legislation that awaits their governors’ approval. While these legislative efforts share common goals, their scope varies. Maryland and New Jersey concentrate on grocery stores, whereas Connecticut’s restrictions broadly encompass retail transactions, and New York’s proposal spans multiple industries.

This legislative trend highlights a growing fear that AI could alter pricing strategies from being supply and demand-based to being centered on individualized predictions. Instead of evaluating the overall product value, AI systems may focus on predicting what a particular consumer might pay. For many Black Americans, whose households possess about 15 cents for every dollar owned by White households, and other low-income groups, this concern resonates deeply. These communities are familiar with the so-called “poor tax,” which leads to paying more over time through means like predatory lending, subprime financial products, and high insurance costs.

The fear is that AI could automate and amplify these disparities, making them less detectable. These systems don’t require direct knowledge of a consumer’s income. Advanced AI can deduce purchasing capabilities and behaviors from purchase histories, browsing activity, location, loyalty programs, device information, shopping frequency, and responses to previous price changes. Individually, these signals may appear minor, but together, they construct detailed consumer profiles capable of predicting behaviors. Algorithmic pricing, unlike traditional inequalities, can create unseen disparities because consumers often only see the final price, without awareness of the data collected or the analysis it undergoes.

Americans have accepted certain dynamic pricing forms, such as fluctuating airline tickets, hotel rates during events, and increased ride-share fares in busy times. These systems traditionally respond to market conditions affecting consumers collectively. Surveillance pricing, however, shifts the focus from the overall market to individual consumers. Maryland’s legislation captures this difference, defining dynamic pricing as varying prices during a business day based on factors like AI systems recalibrating in near real-time.

Surveillance data encompasses information gathered via sensors, cameras, device tracking, biometric monitoring, and technologies capturing consumer behavior, location, or characteristics. At a time when many families face rising food costs and economic pressures, algorithms identifying financially constrained individuals raises questions about digital economy fairness. Concerns are not baseless. Walmart faced backlash over electronic shelf label discussions, leading to fears around real-time grocery price fluctuations. Similarly, Delta Air Lines encountered criticism as AI-driven pricing discussions emerged, sparking concerns over systems identifying maximum consumer payments. Both companies later clarified policies, but the backlash signifies public worry over behavioral data and AI’s economic decision influence.

Once AI systems integrate into daily commerce, regulation becomes challenging. Maryland acknowledges this, noting that enforcing new laws might need technical expertise regulatory agencies lack. Regulators may explore machine learning, behavioral analytics, predictive algorithms, consumer profiling systems, and proprietary software capable of near real-time price recalibration. This poses different challenges compared to those traditional consumer protection laws address. AI decision-making processes are often buried within automated systems, machine learning models, third-party data brokers, and analytics tools operating at a pace consumers cannot access or grasp.

These issues underscore a broader AI misconception, termed “The Miseducation of Technology” — viewing AI as an impartial entity rather than part of the economic and institutional systems it functions within. As the digital economy increasingly values behavioral prediction, consumer surveillance, and profit maximization, AI systems will prioritize these. If profit maximization relies on predicting higher charges, AI might excel at doing precisely that. For generations, economically vulnerable Americans have paid a poor tax. AI did not originate this reality, but it could make it faster, more precise, less detectable, and scalable.

Danielle A. Davis Canty serves as a senior advisor and director of technology policy at the Joint Center for Political and Economic Studies, hosts the “The Miseducation of Technology” podcast, and is a Public Voices fellow with The OpEd Project.