How to pick sources for a niche news digest

Pick sources by measuring them. List candidate outlets with site: searches in Google News, count items per day for a week, and keep the few that add stories nobody else has. That is the source step of newsletter curation, and most guides skip it.

We ran it on vertical farming news. From 2026-10-10 to 2026-10-16 we tested 21 outlets. 13 returned anything in Google News, 8 returned nothing on any of the seven days, and we kept 6.

This is a demonstration list, not a client's subscriber business. How to write the email, pick a format or grow a list is out of scope.

Why gut-feel source lists fail

Two things go wrong. You pick outlets you like and find they mostly republish each other. Or you pick outlets that look active on their own site and find Google News returns nothing for them.

We saw the second one on day one. Of the 21 domains we probed on 2026-10-09, 11 returned zero items in the last 24 hours. During the week, 8 of the 21 returned no item. The first problem was smaller here: only producer.com and foodbev.com had their single item carried by another outlet, and hortidaily.com shared a quarter of its items with Vertical Farm Daily.

The pillar post, How to build a daily news newsletter from feeds, shows the whole pipeline. This post is the source step in detail.

Build a candidate list

We used three routes. Run them for your own niche.

1. Topic searches in Google News. Search 4 or 5 phrases for your niche and note the outlet under each result.
2. Tally the <source> element. Every item in a Google News RSS feed names its outlet. Fetch the same phrases as RSS and count outlets. For the URL formats, see the Google News RSS feed guide.
3. Outlets from outside Google News. Trade sites you already know. Test them anyway, and mark them as found elsewhere.

We picked vertical farming because it mixes trade outlets, regional press and press releases, and the daily volume is low enough for outlets to differ. We used five queries, each with when:7d, US English edition: "vertical farming", "indoor farming", "controlled environment agriculture", "vertical farm" funding and "vertical farm".

The script tally_sources.py (download) fetches them, drops repeated items and counts outlets. Run on 2026-10-09, it found 98 unique items from 27 outlets:

98 unique items, 27 outlets
67	Vertical Farm Daily	www.verticalfarmdaily.com
2	The Packer	www.thepacker.com
2	openPR.com	www.openpr.com
2	Hortidaily	www.hortidaily.com
2	The Western Producer	www.producer.com
2	Kalkine Media	kalkinemedia.com
1	The Conversation	theconversation.com
...

One outlet supplied 67 of 98. Twenty-one of the 27 outlets appeared once. Those single appearances are the noise candidates, and the long tail is where a source list gets padded.

14 of the 21 probed domains came from the tally. We added the other 7 by hand, from trade-press knowledge. We did not verify that any of them is a good outlet; the probe below is the test. The full list is in domains.txt.

Probe each outlet with a site: query

For each domain, run one query with site:, a fixed niche term and when:1d:

curl -s 'https://news.google.com/rss/search?q=site:verticalfarmdaily.com+("vertical+farm"+OR+"vertical+farming"+OR+"indoor+farming")+when:1d&hl=en-US&gl=US&ceid=US:en' \
  | grep -o "<item>" | wc -l

8

Run on 2026-10-09 at about 13:00 UTC. Use grep -o | wc -l, not grep -c. A feed arrives as one long line, so grep -c "<item>" printed 1 for the same response. For site: and when: themselves, see the feed guide, and for counts on other operators see Google News RSS search operators.

The fixed term matters, and our first one was wrong. We started with farming OR agriculture. Five outlets came back with exactly 100 items, which looks like the cap (our earlier feed test saw 100 to 103). One of them, phys.org, returned 100 items that were all from phys.org, so the term was not filtering anything for a broad science outlet. Counted that way, 17 domains had items and they summed to 514 for one day, which is meaningless for a niche this size. We moved the first run to counts_2026-10-09_broad-term.csv and kept it as a warning. With the narrower niche phrases, nothing hit the cap.

Day one with the narrow term, 21 domains:

• 10 returned at least one item in the last 24 hours, 26 items in total.
• verticalfarmdaily.com gave 8, igrownews.com 7, bioengineer.org 4, and seven others 1 each.
• 11 returned nothing, including hortidaily.com and thepacker.com, which both appeared in the topic tally. A domain can show up in a topic search and still return nothing for a site: query with a one-day window.
• global-agriculture.com returned 11 items, none newer than 24 hours, even with when:1d. So when:1d does not guarantee a 24-hour window, and the script checks pubDate itself.

Over the full week, 8 outlets had zero items on every day: globalaginvesting.com, phys.org, kalkinemedia.com, agfundernews.com, growingproduce.com, urbanagnews.com, agritecture.com and thefeedstock.com. Several of them are well known in the niche, but during the week Google News gave us nothing for them with this query and a one-day window. (globalaginvesting.com and phys.org each had one item on 2026-10-09, before the window.) We did not test site:example.com/news path variants or subdomains, so we cannot say whether those would behave differently.

Count items per day for a week

A single day is an anecdote. We fix the query set and run it at the same time every day for seven days.

count_sources.py (download) takes a file of domains, runs one query per domain, counts items whose pubDate is inside the last 24 hours, flags any response at or above the 100-item cap, and appends a row to a CSV. It sleeps 3 seconds between requests, one query per outlet per day. It also saves each fresh item's title to items.csv for the overlap check. We did not test rate limits, so keep the delay.

python3 count_sources.py domains.txt counts.csv

run_utc,domain,items_returned,items_24h,capped
2026-10-09T13:05:37Z,verticalfarmdaily.com,8,8,False
2026-10-09T13:05:37Z,hortidaily.com,0,0,False
...

The full first-day file is counts_2026-10-09.csv.

Results for 2026-10-10 to 2026-10-16 (seven runs at 08:00 UTC), sorted by mean items per day:

Outlet	Mean items/day	Max	Zero days	Unique share
verticalfarmdaily.com	9.14	15	2	92%
igrownews.com	4.71	8	1	100%
hortidaily.com	2.29	9	3	75%
openpr.com	1.57	5	1	100%
bioengineer.org	0.71	4	5	100%
hortibiz.com	0.71	5	6	100%
farmtario.com	0.43	1	4	67%
producegrower.com	0.29	1	5	100%
foodbev.com	0.14	1	6	0%
fruitnet.com	0.14	1	6	100%
global-agriculture.com	0.14	1	6	100%
producer.com	0.14	1	6	0%
thepacker.com	0.14	1	6	100%
agfundernews.com	0.00	0	7	n/a
agritecture.com	0.00	0	7	n/a
globalaginvesting.com	0.00	0	7	n/a
growingproduce.com	0.00	0	7	n/a
kalkinemedia.com	0.00	0	7	n/a
phys.org	0.00	0	7	n/a
thefeedstock.com	0.00	0	7	n/a
urbanagnews.com	0.00	0	7	n/a

Unique share is the share of an outlet's items that no other outlet carried at a headline overlap of 0.6. It is n/a where an outlet had no items. 144 unique items in total. No cell reached the 100-item cap, so nothing is truncated.

Some of this surprised us.

• The mean hides bursts. hortibiz.com has a mean of 0.71, but all of it came on the last day (5 items, zero on the other six). bioengineer.org had 4 on the first day and 1 on the last. They pass a mean rule and still could not feed a daily digest on their own.
• Quiet is not the same as gone. verticalfarmdaily.com returned 12 to 15 items on five days and nothing on 11 and 12 October.
• The feed disagrees with the daily counts. We ran a Feedmarkdown feed over the same 21 queries, without the one-day filter. It gave global-agriculture.com 33 new items in the week, while the one-day count above saw 1. 31 of the 33 had been published more than two days before they arrived. For that outlet, Google News surfaces items days late, so a one-day window undercounts it. We did not look at whether that holds for other outlets.

Keep an outlet only if it adds stories

An outlet that publishes daily is no use if every story is also on another outlet. For each outlet we count the share of its items that no other outlet covered.

overlap.py (download) takes the two CSVs and prints the table above plus the matched pairs. It uses the same headline token-overlap method as dedup.py in the feed guide, with a Jaccard threshold of 0.6. It is a cheap approximation: it misses rewritten headlines and can merge two different stories with similar words. The method flagged 6 pairs across outlets. We read all six. It was wrong on none: five were the same headline on two outlets, and one was a rewritten prefix on the same story. Six is fewer than the ten we planned to check, because there were no more. For where headline matching fails, see the deduplication post.

The decision rule, stated before we saw the week: keep an outlet if its mean is at least 0.5 items per day and at least 30% of its items are unique. Keep one slower outlet if nothing else covers a sub-topic. We did not change the thresholds after seeing the data.

Outlet	Decision	Reason
verticalfarmdaily.com	Kept	9.14 a day, 92% unique. Most of the volume.
igrownews.com	Kept	4.71 a day, all unique. Carries items nobody else did.
hortidaily.com	Kept	2.29 a day, 75% unique. Shares 4 of 16 items with Vertical Farm Daily.
openpr.com	Kept	1.57 a day, all unique. Press releases, so check the tone before you send them on.
bioengineer.org	Kept, watch	0.71 a day, but 4 of 5 items came on day one.
hortibiz.com	Kept, watch	0.71 a day, all 5 items on one day.
farmtario.com	Dropped	0.43 a day, below 0.5.
producegrower.com, thepacker.com, global-agriculture.com, fruitnet.com	Dropped	0.14 to 0.29 a day, below 0.5.
producer.com, foodbev.com	Dropped	Their single item each was also on another outlet.
8 outlets with no items	Dropped	Nothing returned in seven days.

We allowed one slow outlet if it alone covered a sub-topic. None qualified: the Canadian greenhouse story on hortidaily.com was also on Vertical Farm Daily.

Fewer queries, same outlets

In Feedmarkdown a source, which is one query, is what a plan counts. Starter lists 10 sources, Pro 50 and Agency 200 (from the pricing section, checked 2026-10-09). A query per outlet uses them up fast, so test whether a few topic queries without site: return the same stories.

We ran the five topic queries from the candidate step again, with after:2026-10-09 before:2026-10-17 instead of when:7d so they cover the collection week. We compared headlines against the 134 items the six kept outlets produced via site: (topic_coverage.py, download). The after: and before: operators are untested beyond this run.

Query	Items returned	Of the 134 kept-outlet items
"vertical farm"	66	53
"vertical farming"	17	11
"vertical farm" funding	7	7
"indoor farming"	10	3
"controlled environment agriculture"	17	0
All five together		55 (41%)

Five topic queries found 41% of what six site: queries found. Nearly all of it was Vertical Farm Daily (51 of 64). igrownews.com, openpr.com and hortibiz.com came back 0 of 49. The cheap route is a single "vertical farm" query, which alone returned 53 of 134. The rest needs site:. If an outlet matters to your readers and only shows up through site:, spend a query on it.

What a digest from these sources looks like

The six kept outlets averaged 19.1 items a day over the week (134 items in total, from the daily counts). Four of those were duplicates under the headline method, so about 130 unique, or 18.6 a day.

We also ran the real feed. Over the seven counted runs (2026-10-10 to 2026-10-16) the run counters read 738 items found, 80 new, 4 deduplicated, 85 scraped and 4 errors. "Found" includes items already seen on earlier days, which is why it dwarfs "new". The feed rows in the database show 6 duplicate rows, not 4, and we did not chase the difference. We took the first run (2026-10-09) out as warm-up, because it pulled 30 days of backlog: 98 found, 85 new. Costs per run are not in the API response, so we do not quote one.

What we did not test

One niche, one week, the US English edition, Google News as the only index. We did not test paywalled outlets (an item can exist while the body is truncated), outlets that publish weekly, seasonality, or whether readers prefer the kept list. We also did not try Google Alerts as a second route to outlets.

Where Feedmarkdown fits

Feedmarkdown takes a Google News query as a source (Google Alerts is also supported), resolves the redirect links to publisher URLs, removes duplicate stories across sources before scraping, and sends each article as full-text Markdown by signed webhook or as a JSON Feed or Atom pull. That covers the last steps of this post. It does not choose your sources or judge outlets. If an outlet in your list has no Google News coverage, Feedmarkdown cannot pull it today; direct site sources are on the roadmap, and you can ask for one in the FAQ. Plans are on the homepage.