5 min read 841 words 12 views
Designing algorithms for demand you can’t observe

Many marketplace balancing algorithms assume that demand leaves a trace. A user opens an app, runs a search, submits a request or starts a session. Even when no transaction follows, the intent is observable.

That assumption breaks in systems where demand depends on supply being physically present. Scooter rental is a useful example. Most users do not open an app and search for a scooter. They see one on the street, scan its QR code and ride. If no scooter is nearby, the intended trip may leave no observable demand signal in the service’s data.

This is more than a missing-data problem. It changes how balancing systems should be designed. Algorithms cannot rely only on real-time observed demand; they also need proxy signals and structural knowledge about where and when hidden demand is likely to appear.

Why demand is invisible

In scooter rental, only about one-fifth of rentals begin by selecting a scooter in the app. Most happen next to the vehicle itself. This behavior makes sense for last-mile mobility: trips are short and often habitual or impulsive. A commuter may know where scooters usually stand near a station, while another user simply sees one while walking.

Neither behavior requires a search. Consequently, the system observes successful rentals but misses much of the intent that never becomes a trip. A parking lot with zero scooters can therefore look like a location with zero demand, even when users would rent immediately if supply were available.

Finding proxy signals for hidden demand

Latent demand does become visible under some conditions. Large parking lots reveal a recurring pattern. Residential locations tend to lose scooters during the morning commute, while locations near offices, campuses and transport hubs accumulate them. In the evening, the direction reverses.

Designing algorithms for demand you can't observeImage: Author

Conversion from app sessions to rentals provides another signal. In observed data, conversion remained relatively stable outside rush hours but fell by roughly 25 percentage points at a morning-demand location and 40 points at an evening-demand location during peak periods.

Designing algorithms for demand you can't observe
Image: Author

A conversion drop is not proof of unmet demand by itself, but when combined with supply data, it becomes a useful proxy. One plausible mechanism is that users who normally take a scooter directly from the street start opening the app when nearby supply becomes scarce. Previously invisible intent then becomes observable.

High-volume locations are especially useful for identifying these patterns because larger scooter counts reduce noise. Once a recurring morning or evening pattern is established there, it can serve as a prior for smaller locations where a fleet of only a few scooters makes independent estimates unreliable.

Designing for observed and predicted demand

The architectural implication is that a balancing system needs two layers. One reacts to real-time supply-demand signals. The other acts on precomputed knowledge about when and where shortages or surpluses are expected. Several mechanisms can use the same forecast.

Preventive Surge Pricing. Classical surge pricing is reactive: prices rise after the system observes that demand exceeds supply. With latent demand, that signal may arrive only after most scooters have disappeared. Preventive surge instead activates before the expected shortage, using historical patterns or an ML demand forecast for each location.

If a residential parking lot typically runs out at 10 a.m., for example, the adjustment might begin at 8 or 9 a.m., peak around the expected shortage and then decline. The goal is not to respond to a deficit but to reduce its probability and severity before it forms.

Anti-Surge Pricing. The same forecast can identify excess supply. If scooters consistently accumulate near transport hubs after the morning commute, prices can be lowered before they spend hours idle. This can increase utilization of otherwise idle supply.

Predictive Relocation. Relocation has the same problem. A purely reactive algorithm moves scooters after excess supply is observed, but by then another location may already be empty. A predictive system can instead anticipate the flow: after the morning commute, move scooters from transport hubs back toward residential areas; later, reverse the direction ahead of evening demand.

The pattern beyond mobility

The engineering pattern applies wherever absence of supply hides intent. An online store cannot observe purchases of an out-of-stock product, and a content platform cannot observe enrollments in a course it does not offer. In both cases, searches with no results, waitlists, browsing behavior or other proxy signals can expose part of the missing demand.

Engineering takeaways

Latent demand requires a different approach to observability. First, identify moments when hidden intent becomes measurable and use them as proxy signals rather than treating transactions as the complete demand dataset. Second, separate reactive control from predictive control: real-time signals handle current imbalances, while structural forecasts address shortages that have not yet become visible. Finally, use high-signal environments to learn patterns that can cautiously inform low-signal ones.

The broader lesson is simple: when the system cannot observe demand directly, the answer is not necessarily to collect more of the same data. Sometimes the algorithm has to be designed around what the data cannot show.


Featured image credit