Google News links redirect. How to get the real article URL

Every item in a Google News RSS feed links to news.google.com/rss/articles/<id>, not to the publisher, and curl -L never gets from one to the other. This post traces one Google News redirect hop by hop, shows what the final page contains, and covers the two-request method that turns it into the real URL. The Google News RSS feed guide already showed that four links took three hops each and ended on a Google page. That post stopped there. This one follows the link the rest of the way.

We ran everything on 2026-10-11 from the Netherlands, on a home connection (KPN). Google shows a consent step to visitors in the EU, so your hops may differ. The method uses an undocumented Google endpoint. Re-test before you rely on it.

What a Google News link returns

We fetched q=openai+when:1d (100 items) and took the first link. curl -L reported three redirects and a 200. Taking one request per hop with curl -sI shows each step:

The chain is the same for the first three links we tried:

1. 302 from the article link to consent.google.com, setting a SOCS cookie.
2. 303 from the consent host back to the article, with ucbcb=1 added.
3. 302 to the same article with hl, gl and ceid added.
4. 200 on news.google.com/rss/articles/<id>.

No hop goes to the publisher. The last response is a 582,085-byte HTML page titled "Google News". We searched it for the publisher's domain (finance.yahoo.com for this item) and found no match. It has no <meta http-equiv="refresh">. What it does have is three attributes on one element: data-n-a-id (the article id), data-n-a-ts (a timestamp) and data-n-a-sg (a signature). We did not check the page for a JavaScript redirect.

The /read/<id> form of the same article was 1,132,406 bytes, about twice as large, with the same three attributes. That matches a code comment in our resolver library, which uses the smaller /rss/articles page.

Why curl -L never gets there

The HTTP chain ends on a Google page, so the real hop happens in the browser. We opened the same link in Chrome 154 (headless, fresh profile, not signed in to Google) with the DevTools Network panel attached. From the Netherlands a clean browser stops on a consent page, unlike curl, which passed through it on its own. We clicked "Reject all".

After the article page loaded, the page sent three batchexecute requests. The first used rpcids=Fbv4je. The browser then ended on the Yahoo article URL. We did not trace which script performed the final navigation, so we say only that the call is there and the browser lands on the publisher afterwards.

Cookies did not change the outcome. With Cookie: CONSENT=YES+cb; SOCS=CAI curl skipped the consent hop (two requests instead of four) and got the same page with the same three attributes. A resolver does not need the cookie, though. Our Go library sends none and resolved all 300 links in the rate test below, from the Netherlands. It follows redirects the way curl -L does. We did not test from outside the EU.

Can you decode the id yourself?

Not any more, at least for the ids we saw. We took 100 ids from the feed, base64url-decoded each and searched the bytes for http. Zero contained it. All 100 started with CBMi. Each decoded to a short binary header followed by a 150-odd-character ASCII string that begins AU_yqL. Decoding that string again also produced no URL.

Base64-decoding the id is the obvious first try, and you will find scripts that do it. We did not find an id it works on. We cannot say whether an older format carried the URL. We can say that none of the 100 current ids do.

The method that works

It takes two requests. The second is the Fbv4je call the Google News page made in the browser above:

1. GET https://news.google.com/rss/articles/<id>. If the final host is not a google.com host, that is the publisher URL.
2. Read data-n-a-id, data-n-a-ts and data-n-a-sg from the one tag that carries data-n-a-id.
3. POST https://news.google.com/_/DotsSplashUi/data/batchexecute?rpcids=Fbv4je with a garturlreq payload that holds the id, ts and sg.
4. Parse the response. The row for Fbv4je holds a JSON string, and its garturlres item is the publisher URL.

Here it is in standard-library Python. It is the resolver from the pillar post plus a check for a 200 that carries no URL (more on that below):

import json, re, sys, urllib.parse, urllib.request

UA = "Mozilla/5.0"
COOKIE = "CONSENT=YES+cb; SOCS=CAI"  # skips the EU consent redirect

def article_id(link):
    return urllib.parse.urlparse(link).path.rstrip("/").split("/")[-1]

def resolve(link):
    gid = article_id(link)
    req = urllib.request.Request(
        f"https://news.google.com/rss/articles/{gid}",
        headers={"User-Agent": UA, "Cookie": COOKIE},
    )
    html = urllib.request.urlopen(req, timeout=20).read().decode()
    sg = re.search(r'data-n-a-sg="([^"]+)"', html).group(1)
    ts = re.search(r'data-n-a-ts="([^"]+)"', html).group(1)

    inner = (
        '["garturlreq",[["X","X",["X","X"],null,null,1,1,"US:en",null,1,'
        'null,null,null,null,null,0,1],"X","X",1,[1,1,1],1,1,null,0,0,null,0],'
        f'"{gid}",{ts},"{sg}"]'
    )
    body = urllib.parse.urlencode(
        {"f.req": json.dumps([[["Fbv4je", inner, None, "generic"]]])}
    ).encode()
    req = urllib.request.Request(
        "https://news.google.com/_/DotsSplashUi/data/batchexecute",
        data=body,
        headers={
            "User-Agent": UA,
            "Cookie": COOKIE,
            "Content-Type": "application/x-www-form-urlencoded;charset=UTF-8",
        },
    )
    raw = urllib.request.urlopen(req, timeout=20).read().decode()
    row = json.loads(raw.split("\n\n", 1)[1])[0][2]
    if row is None:  # HTTP 200, but no URL in the answer
        raise ValueError("batchexecute returned no URL")
    return json.loads(row)[1]

if __name__ == "__main__":
    for link in sys.argv[1:]:
        print(resolve(link))

We ran it on 50 links from one fresh query, one after another. All 50 resolved, in 29.4 seconds in total (0.59 seconds a link on average, 1.81 at the slowest), with no errors. For each, the publisher host matched the host in the item's <source url>. We requested five of the resolved URLs with plain curl -L: two returned 200 and three returned 403. The 403 came from the publishers' own bot blocking, not from Google. So resolving the URL is half the job. You still need a fetch that the publisher lets through.

Ready-made packages

You do not have to maintain this yourself. WebCrawlerAPI, the company behind the scraper Feedmarkdown uses, publishes the resolver as MIT-licensed open source in two languages.

Go. github.com/WebCrawlerAPI/google-news-go (v0.1.0) is the library our Google News source calls. Its README gives this install line:

go install github.com/WebCrawlerAPI/google-news-go/cmd/google-news-go@latest

We built the CLI from the same module's source rather than running that exact line, then ran it on the first link. It printed the Yahoo URL as JSON with "method": "google-news-rpc". In Go code:

res, err := googlenews.Resolve(ctx, "https://news.google.com/read/ARTICLE_ID")
if err != nil {
	log.Fatal(err)
}
fmt.Println(res.URL)

On 10 links it returned the same URLs as the Python script above, 10 of 10.

JavaScript. github.com/WebCrawlerAPI/google-news-url-resolver is the source of the npm package @webcrawlerapi/google-news-url-resolver. Its README says Node.js 20.3+, no dependencies, and gives:

npm install @webcrawlerapi/google-news-url-resolver

import { resolveGoogleNews } from '@webcrawlerapi/google-news-url-resolver';

const { url } = await resolveGoogleNews(
  'https://news.google.com/rss/articles/CBMi...?oc=5',
);

It also has a resolveMany helper with concurrency and delayMs options, and typed error codes such as RATE_LIMITED, NO_SIGNATURE and RPC_EMPTY. Version 0.1.0 went up on npm on 2026-10-11. We tested the same code from a clone of the repo. resolveGoogleNews returned the Yahoo URL for the first link, and resolveMany with concurrency: 2 and delayMs: 250 matched the Python results on 10 of 10.

What fails

Failure	What we saw	Status
Malformed link, or a link already on a publisher's site	Go library: Expected an HTTPS news.google.com read/articles/rss/articles URL, before any request	Observed
Made-up or truncated id	HTTP 400 from news.google.com (Go), HTTP Error 400: Bad Request (Python)	Observed
Missing, wrong or zeroed sg/ts	HTTP 200, 108 bytes, with ["wrb.fr","Fbv4je",null,null,null,[3],"generic"] and no URL	Observed
Consent page or blocking, so no sg/ts	Go library returns "Missing Google News signature/timestamp (consent, blocking, or markup change)"	In the code only. We did not trigger it
Rate limit	Documented as HTTP 429. We did not see one	Not observed
Google changes the call	The README warns about it	Not observed

The third row is the one that bites: a 200 does not mean success. We sent the POST with an empty sg, with ts set to 0, with a wrong sg and with a ts one day old. All four came back 200 with no URL. The earlier version of the script would have crashed on that with TypeError: the JSON object must be str, bytes or bytearray, not NoneType, which is why the version above checks for it.

We did not test the Python script on a link that already points at a publisher. The Go library rejects it, since it accepts only news.google.com URLs.

Rate limits and caching

We resolved distinct ids with the Go library at three concurrency levels. Each run used 100 ids no earlier run had touched, with a cool-down between runs.

Concurrency	Requests	Succeeded	Errors	Wall time	Median per link	95th percentile
1	100	100	none	11.3 s	111 ms	139 ms
4	100	100	none	3.4 s	120 ms	178 ms
16	100	100	none	1.9 s	282 ms	515 ms

That was 300 resolves, plus the 50 above, from one home IP in under ten minutes. We saw no 429, no consent page and no block, so we found no limit. We stopped at 16 on purpose: this is Google's undocumented endpoint and we did not want to push it. Do not read the table as a safe rate. A datacentre IP or a longer run may behave differently, and we did not test either.

Our adapter caps resolves at 4 at a time and skips ids it has already resolved. That cap comes from caution and a comment in the code that says Google's resolve call "rate-limits aggressively". It is not a number Google published, and our run does not support or contradict it.

Caching is cheap insurance. We resolved the same id twice and got the same URL both times. We also re-sent a ts/sg pair taken from an earlier page fetch, at 105 seconds and again at 7 minutes old. Both still returned the URL. We did not test a pair a day old, so we do not know how long a pair stays valid. Cache by the feed item's GUID and you avoid the question.

Other Google redirect links

We scanned 9 feeds on the day: top stories, two searches, a site: search, the Technology and World topics, and German, British and Indian editions. That was 680 items. All 680 links were news.google.com/rss/articles/CBMi.... We saw no /goto links and no CAES tokens, so we cannot say how to handle them.

Google Alerts feeds use a different redirect. Their google.com/url?... links return a 200 with the publisher URL readable in a url= parameter, so no resolver call is needed. Our Google Alerts RSS feed post has the details. For how often these links show up in practice, see Is there a Google News API: in its run, all 82 links were redirect links.

What we do in Feedmarkdown

Feedmarkdown's Google News source resolves news.google.com links to publisher URLs before dedup and scraping. It resolves four at a time and skips ids it already has. Items that fail to resolve are dropped from that run, so a resolve outage looks like an empty run rather than an error. The result arrives as full-text Markdown by webhook, or you pull it as JSON Feed 1.1 or Atom. Google News and Google Alerts are the live sources today. Start on the home page or see the pricing.