Pricing a whole Steam inventory when Steam won't talk to your server

We run a small free tool: paste a Steam profile link, and it tells you what the items in that inventory are worth on the Steam Community Market. No sign-in. Under the hood that means reading a public inventory, finding a market price for every item, and adding them up.

The web app is Next.js running on Cloudflare Workers (through OpenNext), with Supabase Postgres as the database. For the first version, the Worker asked Steam for prices directly while the visitor waited on the valuation page.

Then, on 19 October 2026, Steam started answering requests from our Cloudflare Worker with HTTP 429. Not now and then. Every time. Price lookups stopped. So did Steam sign-in verification, or so we thought at first.

This post covers what we changed: a price crawler that runs inside Postgres, a cache that means most items are already priced before anyone asks, back-off rules for two kinds of rate limit, the arithmetic behind "what you actually receive after Steam's fees", and a lesson about counting our own accounts.

Moving the requests somewhere else

We couldn't make Steam accept requests from Cloudflare. But the database lives elsewhere, and Postgres can make HTTP requests with the http extension. Supabase also offers pg_cron. So the crawler became a set of SQL functions, scheduled every minute:

create extension if not exists http with schema extensions;
create extension if not exists pg_cron;

select cron.schedule('steam-price-crawl', '* * * * *', $$select public.crawl_tick()$$);

The Worker no longer asks Steam for market prices at all. Its job shrank to one question: does every item in this valuation have a price yet?

For the lookups the crawler uses the Steam Market's search endpoint, /market/actions?q=Search. One request returns up to 100 items with their lowest sell price in US dollars. That's a lot better than one request per item. Each result is saved into a price_cache table keyed by (appid, market_hash_name).

Two small defensive choices in the save function:

where coalesce((r->>'eCurrency')::int, 1) = 1   -- only USD prices; don't guess other currencies
  and steam_price_cents(r->>'strMinSellSubtotal') > 0

If Steam ever returns a price in another currency, we skip it rather than guess. And when the crawler searched for one specific item and it isn't in the results, it records that item as "not on the market" (source = 'none') so it isn't searched again every minute.

These functions make outbound requests, so they must never be callable by website visitors through the Supabase API. The migration revokes execute rights from anon and authenticated on every one of them.

What to crawl first

Each crawl_tick() does two kinds of work.

Items someone is waiting for. A function called crawl_wanted lists items in valuations that are still being priced and have no recent price. Priority (VIP) valuations come first, then the rest in order of creation. Items whose lookups failed three times in the last hour are skipped for now, so one broken name can't eat the whole budget.

The whole catalogue, in rotation. When nobody is waiting, the crawler pages through each game's full market listing. Each game has a refresh interval:

The next game to crawl is the one most overdue relative to its interval:

order by (cycle_started_at is null) desc,
         extract(epoch from now() - cycle_started_at) / (refresh_hours * 3600) desc nulls first,
         sort

The catalogue crawl is what makes the tool feel fast. A comment in our migrations notes that the database usually already holds more than 95% of an inventory's prices by the time someone asks. So a single-item lookup is only for the stragglers.

Two rate limits, two responses

The first rule is the obvious one. Any 429 sets a 15-minute pause:

if st = 429 then
  insert into rate_limits (key, until) values ('crawler', now() + interval '15 minutes')
  on conflict (key) do update set until = excluded.until;
end if;
exit when st is distinct from 200;   -- any other error: try again next minute

Every request goes into a crawl_log table, which turned out to be the most useful thing we built. It showed the second limit. Steam also rations requests from Supabase's addresses. At 8 requests a minute we got about five hours before being blocked for about seven. During those seven hours, customers' items couldn't be priced either.

The fix was to stop spending the budget on the catalogue. A new setting caps catalogue pages at 2 per minute by default, and everything else goes to items people are actually waiting for:

for i in 1 .. s.crawl_per_tick loop
  st := crawl_item(fx);              -- someone's item first
  if st is null then                 -- nobody waiting
    exit when pages >= s.crawl_pages_per_tick;
    pages := pages + 1;
    st := crawl_page(fx);            -- then one catalogue page
  end if;
  ...
end loop;

We also widened what counts as fresh. A cached price now stays valid for 7 days. Single-item lookups only run for valuations still being priced. If an item still has no price when the valuation's time is up, it's marked "check next time" and the valuation finishes anyway. Priority valuations wait at most one minute. A second pg_cron job closes valuations whose time has come, so they finish even if nobody has the page open.

A detour: not every failure was the block

Sign-in through Steam (OpenID) also broke that day, and it was easy to blame the 429s. We added a fallback that sends the verification request through the database too. But the real cause turned out to be different. Steam decoded %2F in our return_to URL when sending it back, so the signed value no longer matched. The fix was a return_to with no query string, with the page to return to kept in a cookie. Steam's verification endpoint also refused requests without a User-Agent header.

The lesson: when one big failure is going on, check each symptom on its own before you file it under the big one.

"What you actually receive" after Steam's fees

The headline number most people want isn't the market price. It's what lands in their Steam Wallet if they sell. Steam takes two cuts from the price a buyer pays: a 5% Steam fee and a 10% game fee, each at least one cent. Fees are worked out from the seller's amount, so going backwards from a buyer price needs a small search:

/** Seller receives: buyer pays `buyerCents`; Steam takes 5% (min 1 cent) + game 10% (min 1 cent). */
export function sellerReceives(buyerCents: number): number {
  const fee = (s: number) => s + Math.max(1, Math.floor(s * 0.05)) + Math.max(1, Math.floor(s * 0.1));
  if (buyerCents < 3) return 0;
  let s = Math.max(1, buyerCents - 2);
  while (s > 1 && fee(s) > buyerCents) s--;
  while (fee(s + 1) <= buyerCents) s++;
  return s;
}

It finds the largest seller amount whose total with fees still fits in the buyer's price. Below 3 cents nothing can be sold, because the two minimum fees alone take 2 cents.

We also don't assume an item sells at exactly the lowest listing. To sell reasonably soon you'd list a little under it. So the "you receive" figure is based on a price about 1% below the lowest listing, at least 1 cent lower and at most 100 cents lower, and then run through sellerReceives:

export function listPrice(lowestCents: number | null | undefined) {
  if (!lowestCents || lowestCents < 1) return null;
  const buyer = lowestCents - Math.min(100, Math.max(1, Math.round(lowestCents / 100)));
  const seller = sellerReceives(buyer);
  return seller >= 1 ? { buyer_cents: buyer, seller_cents: seller } : null;
}

An exchange rate from Steam's own prices

The search endpoint gives US dollar prices when you aren't signed in, and the crawler stores prices in Malaysian ringgit. We needed an exchange rate, and we wanted it to match how Steam itself converts.

So we ask Steam. We take one item that trades constantly, the Team Fortress 2 "Mann Co. Supply Crate Key", and fetch its price overview twice: once in USD, once in the local currency. The rate is the ratio of the two medians, falling back to the lowest price if a median is missing:

const ref = { appid: 440, name: "Mann Co. Supply Crate Key" };
const usd = await priceOverview(ref.appid, ref.name, 1);
const loc = await priceOverview(ref.appid, ref.name, currency);
const u = usd?.median_cents ?? usd?.lowest_cents, l = loc?.median_cents ?? loc?.lowest_cents;
return u && l ? l / u : null;

The crawler refreshes the rate every 12 hours. After a failed attempt it waits 30 minutes before trying again, so a bad hour doesn't add two wasted requests to every minute. If there's no rate at all, the tick does nothing rather than store prices converted with a made-up number.

Lesson: don't count yourself

The site also has an optional paid consignment service. We tested it with our own Steam account. That account sits in two lists in the admin settings: VIP accounts (free priority pricing) and test accounts (charged only US$0.50 for that service, the minimum Stripe will take). The payment we made to check that Stripe live mode worked was our own US$0.50 card payment.

That's fine for testing. But it means our own activity looks exactly like a customer's: valuations, sign-ins, orders, payments. A dashboard that counts every row will count us, and on a quiet day "us" can be most of the numbers.

So when we added a 30-day funnel to our internal dashboard, own accounts were excluded from the start, read from the same two settings lists rather than from anything hard-coded:

export function summarizeValuations(vals, logins, orders, own: string[], now: number) {
  const mine = new Set(own);
  const v = vals.filter((r) => !mine.has(r.steamid));
  const o = orders.filter((r) => !mine.has(r.steamid));
  // ...counts, sources, sign-ins, orders, paid orders, listings
}

The endpoint returns only counts and short source labels. No Steam IDs, no names. If you're building something with test payments, decide early how your own accounts are marked, and filter them out in one place.

Try it

The valuation tool is free and needs no sign-in: https://steam.2cstudio.app. Your inventory needs to be public. If an item shows "check next time", the crawler just hasn't reached it yet.

Written by 2C Studio, a small studio.