# How to Scrape Hotel Prices Without a Browser: Reverse Engineering Radisson's Request Flow A browser is the easy way to scrape a site protected by bot detection. It is also the slowest and the most expensive one. In this post we build the other kind of scraper for [Radisson Hotels](https://www.radissonhotels.com): one that sends only the HTTP requests that return the data, and gets past Akamai Bot Manager without ever running a browser. Radisson is a good test case for hotel price scraping. The prices aren't in the HTML, they come from a JSON API that the page calls after it loads, and the site runs three of Akamai's checks before it lets those calls through. We'll show how we got there. We recorded one real browser session, found the four API calls that matter among 148 requests to Radisson, worked out which of Akamai's checks the site runs and where, and rebuilt the whole flow in Go. Then we ran it through [Massive](https://joinmassive.com) residential proxies in the UK. > **Key Takeaways** > > - Radisson's search results come from four JSON endpoints. Two of them, the city lookup and the rates call, are where Akamai blocks a bad session. > - The site runs three Akamai checks on page load: SBSD, the `_abck` sensor and a pixel challenge. A request-based scraper has to answer all three, in the order the page does. > - Over 100 sessions through Massive UK residential IPs, every session cleared Akamai, and 490 of 500 planned searches returned rates (98%). > - Clearing Akamai cost 520 KB of proxy traffic per session, and three quarters of that was two JavaScript files. A search after that cost 12.5 KB. > - A session that clears Akamai keeps working. Five sessions ran 100 searches each without being stopped once, at 18 KB per search all in. ## Why request-based A request-based scraper skips everything a browser does that the scraper doesn't need: rendering, images, fonts, maps, analytics and the page's own JavaScript. What's left is a handful of HTTP requests, so it's faster, it runs hundreds of sessions on one machine, and it moves far less traffic through your proxies. The catch is that you have to know exactly which requests to send. Bot detection doesn't only look at what you request. It looks at how: the TLS handshake, the order of the HTTP/2 headers, the cookies and the JavaScript challenges a real browser answers in the background. So the work is in reverse engineering the request flow first. ## Step 1: Record a real session We started by doing a search on Radisson in Chrome, with the traffic going through [powhttp](https://hypersolutions.co/blog/how-to-use-powhttp-web-scraping), an HTTP debugging proxy. We opened the home page, typed "Amsterdam", picked dates and searched. We use powhttp rather than the browser's DevTools for one reason. DevTools shows you a cleaned up version of each request. powhttp shows what actually went over the wire: the real header order, the HTTP/2 pseudo-headers, the TLS fingerprint and every cookie header the browser sent. Those details are exactly what bot detection checks, and what a scraper has to copy. ![powhttp request list for the recorded Radisson search, filtered to hosts ending in radissonhotels.com](https://cdn.sanity.io/images/6hf2slow/production/bdd569f92c8c3614f766f6f5d35e8a570f104f74-2316x1224.png "The recorded search, filtered to Radisson's own domains: the page, its scripts and images, Akamai's endpoints and the API calls.") That one search produced 568 requests to 46 hosts. Most were noise: Google Maps, analytics, a consent banner, images. Filtering on Radisson's own domains left 148 requests: JavaScript bundles, images, Akamai's scripts, and 38 calls to JSON APIs. ## Step 2: Find the requests that return the hotel prices The hotel prices weren't in the HTML. The results page is a JavaScript application that loads them after it renders, so we went through the 38 API calls and looked at what each one returned. Four of them carry everything a search needs: - `GET /zimba-api/autocomplete/amsterdam` returns the place you typed, as a Google place ID. - `GET /zimba-api/search-result/hotels/search?placeId=…` returns the codes of the Radisson hotels for that place, with distances. - `POST /availability-api/multiple-hotels-rates` returns the cheapest rate of each hotel for the dates and guests. - `GET /zimba-api/search-result/hotels?code=…` returns a hotel's name, address, stars and review score. The rates call is the one that matters. It takes the dates, the guests and the list of hotel codes as a JSON body, and returns a price per hotel, the total with tax, and how many rooms are left. ![powhttp showing the multiple-hotels-rates request body and its JSON response](https://cdn.sanity.io/images/6hf2slow/production/44408488a04a170bf57d722821d932ab49236319-1200x1555.png "The rates call: dates, guests and hotel codes go in, a price per hotel comes out.") The other 34 calls load labels, currencies, exchange rates, phone numbers and loyalty content for the page. A scraper doesn't need any of them. ## Step 3: Copy the requests exactly Next we looked at the headers of the four calls. Most are what any Chrome fetch sends. A few are specific to Radisson: - `x-source-of-business: desktop` on every call, and `x-version: V2` and `x-currency` on the rates call. - `transactionid`, a 40-character value that's the same on every call in one visit. The `transactionid` looked like it could be a token from the server. It isn't. Searching the site's JavaScript in powhttp for `transactionId` found the function that makes it: 40 random digits in base 15, created on the first page load and kept in the browser's session storage. So our scraper generates one per session the same way. ![powhttp search results for StoreTransactionId, open on home.js at the function that generates the transactionid](https://cdn.sanity.io/images/6hf2slow/production/573e831ceba36a2415fef8927409e3124942d812-2881x1550.png "The transactionid is 40 random base-15 digits, made in the browser and kept in session storage.") Then we copied the header order. Chrome sends headers in a fixed order that differs by request type: a page load, a script, a `fetch()` and a form post each have their own. We copied each order from the recording, including where the cookie header goes. For the connection itself we used [tls-client](https://github.com/bogdanfinn/tls-client) for Go, which opens connections with Chrome's TLS and HTTP/2 fingerprint. The standard Go HTTP client has a TLS fingerprint no browser has, and the TLS fingerprint is one of the first things bot detection looks at. ## Step 4: Find where Akamai sits With the data requests known, the next question was what Akamai needs before it lets them through. The answer is in the home page HTML and in the requests the page makes right after it loads. The page includes three Akamai scripts: - **The sensor script**, at a long random-looking path such as `/hLJOkaPJ/…/XMmkuCU0`. It collects data about the browser and posts it back to the same path, three times. Each answer updates the `_abck` cookie, and once the cookie contains `~0~`, Akamai considers the session valid. - **The SBSD script**, at a similar path with a `?v=` parameter. SBSD (State Based Scraping Detection) is a second Akamai check. The page posts payloads to the same path while it runs, three times in our recording, and each answer updates the `bm_s` cookie. Our scraper posts two, which was enough. - **The pixel script**, at `/akam/13/…`, with a number written into the page. The page answers it once with a form post to the same path with `pixel_` in front. ![The Radisson home page HTML with the Akamai pixel number in the head and the sensor and SBSD script tags at the end of the body](https://cdn.sanity.io/images/6hf2slow/production/7b219dad0603443a4d9d6a656dd55942b76e1a2d-2015x536.png "The home page loads Akamai's sensor script and its SBSD script from random-looking paths, and writes the pixel number into the head.") ![powhttp showing a sensor POST with status 201, a success response and the _abck cookie it sets](https://cdn.sanity.io/images/6hf2slow/production/bc4d033eff1c203a0e75bef2d5808e1b304abcfd-2015x783.png "A sensor post: Akamai answers 201 and sets an _abck cookie containing ~0~.") Two details matter here. The script paths change, so a scraper has to read them from every home page it loads rather than hardcode them. During our tests the SBSD path changed several times. And on some IPs, the home page isn't the home page at all. It's a short SBSD challenge page with a `t=` parameter on the script. The scraper has to answer that challenge and load the page again before it can do anything else. None of these checks can be passed by copying a recorded payload. Each one is generated from the script, the cookies, the user agent and the IP of the current session. We generate them with the [Hyper Solutions API](https://hypersolutions.co/products/akamai): the scraper downloads the script through its proxy, sends it to the API with the session's cookies and IP, and posts the payload the API returns. We used our [Go SDK](https://github.com/Hyper-Solutions/hyper-sdk-go) for that. How each of these checks works on its own, with SDK code in Go, Python and JavaScript, is covered in our [guide to Akamai without a headless browser](https://hypersolutions.co/blog/bypass-akamai-without-headless-browser). ## Step 5: Check your scraper against the browser Before scaling up, we pointed the scraper at powhttp too, so its requests landed next to the browser's in the same tool. That makes the comparison direct: - The TLS fingerprint matched Chrome's. The JA4 hashes of the browser and the scraper differed only in the random GREASE value Chrome inserts on purpose. - The header order matched on every request type. - The cookie header was split into one header per cookie, as Chrome does over HTTP/2. ![The browser's and the scraper's rates requests side by side in powhttp, headers in the same order](https://cdn.sanity.io/images/6hf2slow/production/25f8a22d0a860b08e005651a8ad47dacc2b1aec6-2881x1240.png "The browser's request (left) and the scraper's (right): the same headers in the same order, only values like the dates and currency differ.") If something is off, this is where you find it, and it's much cheaper to find it here than in a block rate later. ## The finished flow Put together, one scraping session does this: 1. Look up its own exit IP, because the Akamai payloads are generated for it. 2. Load the home page. If it's the SBSD challenge page, answer the challenge and load it again. 3. Download the SBSD script and post two SBSD payloads. 4. Download the sensor script and post sensor data until `_abck` contains `~0~`, answering the pixel challenge after the first post. 5. Search: look up the city once, then for each search fetch the hotel list and post the rates request. A session can keep searching after that. If Akamai answers a protected call with a challenge, the session answers it and retries once. The code is open source: [github.com/Hyper-Solutions/radisson-scraper](https://github.com/Hyper-Solutions/radisson-scraper). It's a Go package and a small command-line tool that prints the hotels and rates for a city as JSON:
BASH
git clone https://github.com/Hyper-Solutions/radisson-scraper.git && cd radisson-scraper
export HYPER_API_KEY=your-key
export PROXY_URL='http://your-username-country-GB-session-{session}-sessionttl-30:your-password@network.joinmassive.com:65534'
go run ./cmd/radisson -city london -checkin 2026-11-20 -checkout 2026-11-22 -currency GBP
## Setting up the Massive proxies Every session needs its own residential IP, and it has to keep that IP until it's done. Akamai's payloads are generated for one IP, so if the IP changes in the middle of a session, the session is no longer valid. Massive's dashboard has a Proxy Generator that builds the connection string for you. We picked residential proxies over HTTP and set the rotation type to Sticky, with a session TTL of 30 minutes. ![The Massive Proxy Generator set to residential proxies over HTTP, with sticky rotation and a 30-minute session TTL](https://cdn.sanity.io/images/6hf2slow/production/50c42e181ca54839cf29788fc38d7390fecdb58d-1403x1167.png "Sticky rotation keeps one IP per session ID, here for 30 minutes.") Then, under Geotargeting, we picked the United Kingdom. The generator puts all of it in the proxy username: ![The Massive Proxy Generator with the United Kingdom selected and the generated cURL command showing the sticky GB username](https://cdn.sanity.io/images/6hf2slow/production/b762ea2745768cb53b30ef0c29359fe77b35f9b8-921x1200.png "With the United Kingdom selected, the username becomes your-username-country-GB-session-ID-sessionttl-30.") - `country-GB` picks UK residential IPs. - `session-` makes the session sticky. Every request with the same ID leaves from the same IP, and a new ID gets a new IP. The generator fills in a random ID; our scraper puts a new one in for each session. - `sessionttl-30` keeps that IP for the session for 30 minutes, longer than any of our sessions ran. ## The results We ran 100 sessions through Massive UK residential IPs, 10 at a time. Each session cleared Akamai and then ran five searches, each for a random city from a list of 16 and random dates two to 12 weeks out. A counting proxy on our machine measured every byte between the scraper and Massive. We ran the test on October 22, 2026. - **Sessions that cleared Akamai:** 100 of 100. - **Sessions that completed all five searches:** 98 of 100. - **Searches that returned rates:** 490 of 500 planned (98%). - **Median time to clear Akamai:** 2.6 seconds. - **Median time per search:** 1.5 seconds. The two sessions that failed both cleared Akamai and were then refused on their first rates call. Both were blocked for good: clearing Akamai again on the same IP didn't help. That's an IP Akamai doesn't trust, and the fix is a new session on a new IP. ### What it costs in proxy traffic ![Bar chart of the request-based scraper's proxy traffic per step: the Akamai bootstrap, the city lookup, one search and the IP lookup](https://cdn.sanity.io/images/6hf2slow/production/ab3e852b1577a97a6812ec9bd3221de0224e8c21-2400x1164.png "Mean proxy traffic per step through Massive UK residential IPs.") Clearing Akamai, which we call the bootstrap, is by far the most expensive part of a session: 520 KB. A search costs 12.5 KB: 5.2 KB for the hotel list and 7.2 KB for the rates. Looking up a city costs 6.4 KB, once per city. ![Bar chart of the bootstrap's proxy traffic by step, with the SBSD and sensor scripts as the two largest](https://cdn.sanity.io/images/6hf2slow/production/2e57c4b745fb13ef563d19d5771a422d4fe39187-2400x1764.png "Mean proxy traffic per bootstrap step.") Almost all of the bootstrap is two downloads: the SBSD script (189 KB) and the sensor script (195 KB). Together they're 74% of it. The home page is 64 KB, and all the posts together are under 60 KB. With five searches per session, that works out to 122 KB of proxy traffic per search, everything included. The longer a session lives, the less the bootstrap weighs. In a second test, five sessions each ran 100 searches on a single bootstrap. All 500 searches returned rates, Akamai never stopped any of them, and the traffic came to 18 KB per search, everything included. ## Practical tips - **Record with a tool that shows the wire.** Header order, HTTP/2 pseudo-headers and the TLS fingerprint aren't visible in DevTools. They're what bot detection checks. - **Look for the JSON, not the HTML.** If a page is a JavaScript application, its data comes from an API. Calling that API directly is cheaper and more stable than parsing the page. - **Read the site's JavaScript before you guess.** Values like `transactionid` look like server tokens and turn out to be random. A search through the bundle answers that in minutes. - **Parse Akamai's script paths every time.** They change. - **Verify your scraper in the same tool as the browser.** A side-by-side comparison finds a wrong header order or fingerprint before it shows up as a block rate. - **Keep one IP per session.** Use sticky sessions, check the exit IP, and start a new session on a new IP when one gets blocked. - **Reuse sessions.** The bootstrap is the expensive part, so run as many searches on one session as the site lets you. ## Frequently Asked Questions ### Can you scrape hotel prices without a headless browser? On Radisson, yes. The prices come from a JSON rates API. A scraper that sends only the page, Akamai and API requests, with Chrome's TLS fingerprint and header order, returned rates for 490 of 500 planned searches in our test. ### Why use sticky residential proxies for hotel scraping? Akamai's payloads are generated for one IP. If the IP changes in the middle of a session, the session is no longer valid, so each session needs to keep one residential IP until it's done. ### How much proxy traffic does a request-based hotel scraper use? In our test, clearing Akamai cost 520 KB per session and each search 12.5 KB. With five searches per session that's 122 KB per search all in; with 100 searches on one session it dropped to 18 KB per search. ## Methodology notes - **Dates and scale.** We ran the tests on October 22, 2026, through Massive UK residential IPs: 100 sessions, 10 at a time, five searches each. The longer test ran five more sessions, five at a time, with 100 searches each. - **Searches.** A search is the hotel list for a place plus one rates request for all of its hotels, for two adults and one to three nights. The hotel content (names and addresses) changes rarely, so we left it out of the per-search cost. - **What counts.** Traffic is what our counting proxy saw between the scraper and Massive, including the connection setup, TLS handshakes and failed attempts. The calls to the Hyper Solutions API went straight from our machine, not through the proxy, so they're not counted. - **Browser identity.** The scraper presents itself as Chrome 155 on Windows, with the headers and header order of the recorded session. ## Run it yourself - **Code:** the [radisson-scraper repository](https://github.com/Hyper-Solutions/radisson-scraper) has the full Go flow. - **Proxies:** set up a sticky residential session in [Massive's](https://joinmassive.com) Proxy Generator as shown above. - **Akamai payloads:** the [Hyper Solutions Akamai API](https://hypersolutions.co/products/akamai) generates the sensor, SBSD and pixel payloads; start with the [Akamai getting started docs](https://docs.hypersolutions.co/akamai-web/getting-started). Every new Hyper Solutions account starts with a free one-week trial.