How to get an answer out of this

The 60-second version

Three ways in, in the order most people should try them:

1. A prepared query. Twenty saved questions with their caveats written into them. Each has a search box at the top. Change the word, press run. Start here.
2. A table. Every table is browsable, filterable and exportable. Good when you know roughly what you are looking for and want to poke at it.
3. Ask me. If a question is worth answering it is worth me writing the query properly, and I would rather do that than have a number misquoted. [email protected].

Working the interface

The site is Datasette. Five things carry almost all the value:

• The search box on a saved query is a real parameter, not a keyword search. Typing meta where it says vendor runs the query for that vendor.
• Facets are the grey links across the top of a table. Clicking state or source shows counts per value and lets you filter to one with a click. This is the fastest way to see the shape of a table you have never opened.
• Column filters sit under the table name. contains is usually what you want; exact matches the whole cell.
• Export is at the bottom of every result: CSV and JSON. The CSV honours your filters, so filter first, then export.
• The URL is the query. Everything you click is in the address bar, so you can paste a link to a colleague and they see exactly your result. Adding &_size=max returns more rows per page.

A result page that spins for a while on first load is normal; the snapshot is large and the first query after a rebuild has to warm up. A second query is fast.

Start from your question

"What did my city pay this company?"

Vendor awards, contracts and payments — search any vendor name across federal and local records.

Before you quote a total, run the same vendor broken out by record type. This table mixes things that are not the same: a contract ceiling, an approved award and a cash payment all look like dollars. Summing them is the single most common way to get a number wrong here, and it is why the breakdown query exists.

For one city's own payment file, New Orleans is loaded at line-item grain: what a New Orleans public body paid a vendor. Try amazon, then meta — the contrast is the story in that dataset.

"How did my council vote on this?"

How a town's council voted — per-member roll calls where the town publishes them.

The honest caveat: most local governments do not publish machine-readable roll calls. Where we have them they are real and per-member; where we do not, you will see the decision but not who voted which way. Absence here means the town did not publish it, never that the vote did not happen.

"Who is lobbying, and what did they spend?"

Who is registered to lobby for a company (Louisiana), and what lobbyists reported spending on named officials.

The trap that will bite you: Louisiana expenditure reports are filed by the lobbyist and name no client. A contract lobbyist carries dozens of principals and reports one meal once. So "Company X bought Legislator Y dinner" is not supported by this data. The supportable sentence is "lobbyists registered for X reported $N on Y, across all of their clients."

"Where is the money in politics here?"

Largest campaign contributions to a recipient, and federal PAC money into one state.

Resolve a PAC by its FEC committee ID, not its name — committees get renamed and a name match silently loses years. Meta's is C00502906, which filed as FACEBOOK INC PAC before 2022.

"Which towns are pushing back on data centers?"

Towns with a recorded vote adopting a data center moratorium.

This one is a good illustration of how these numbers are built, so it is worth a paragraph. A first pass produced 97 towns. Checking each against the town's own minutes cut it to 35: 51 rows turned out to quote an agenda-item title rather than an outcome, nine referred to a moratorium adopted somewhere else, and two were a committee recommendation and a first reading. What survives is only rows carrying a vote tally, "unanimous" or "motion carried". Treat it as a floor — "at least 35" — not a census.

And a moratorium is a pause, not a rejection. These towns paused accepting applications while they write zoning rules. Writing that they "rejected" or "voted down" data centers is wrong and a town will tell you so.

"What is actually in the agenda packet?"

Search the staff-report text — full text of the attachments behind agenda items, where unit counts, dollar figures and developer names usually live. This is often where a story is, because the summary rarely carries the number and the attachment does.

"How are the schools doing?"

A Texas district's STAAR scores over time, its spending by category, and board decisions next to proficiency. STAAR and PEIMS are Texas only; school_stats is national.

"What did police buy?"

For a company, start with the surveillance-vendor timeline. Replace flock with axon or motorola; add a city, a two-letter state, or a date such as 2024-01-01 only when you want to narrow it. For a device whose vendor may be unknown, use the technology search. Try ALPR, face recognition, gunshot, CellHawk, or StingRay separately; aliases matter.

A useful example is Rialto, California: the index points to a September 2024 agenda item for a $930,005.16 radio-frequency detector/cell-site simulator. That is a lead, not a publishable conclusion. Open the linked agenda, confirm the vote or action, identify the actual manufacturer (the named contracting party is not necessarily the manufacturer), and ask the city what it bought and under what policy.

How to work a result: first scan cities and dates; then read amount_type and dup_rank; then open source_document and verify the quote in context. Filter before using the CSV link at the bottom. Keep ceilings, proposals, renewals, amendments, and actual payments separate. For an exhaustive vendor search, repeat it with legal names and product brands—for example Motorola Solutions plus Vigilant, or LeadsOnline plus CellHawk.

If you prefer SQL, open the database's SQL editor and use this pattern:

SELECT city, state_norm AS state, spend_date, vendor, product,
       amount, amount_type, dup_rank, quote, source_document
FROM surveillance_spending
WHERE lower(vendor) LIKE '%flock%'
  AND (state_norm = 'CA' OR 'CA' = '')
ORDER BY state, city, spend_date;

Change flock and CA; delete the state line for a national search. Do not write sum(amount) until you have manually resolved duplicate procedural stages and separated ceilings from stated amounts.

For a high-level check, use the conservative surveillance total and, next to it, everything wrong with that table, counted. Read the second before quoting the first. That pair exists because a published figure from this corpus was once 2.9x too high, and the fix was to publish the floor and the defect list together.

Five rules that keep you out of trouble

1. A row count is not an event count. One procurement or one meeting can appear in several rows and in more than one table. count(*) counts rows.
2. Do not sum a column because it holds dollars. Ceilings, authorisations, proposals and payments live side by side in several tables. Break out by record type first.
3. A band is not a number. Several disclosure regimes publish ranges ("$25,000–$49,999", "$1,000,000 or more"). Those are stored verbatim and must never be parsed into a figure or added up.
4. Absence is usually the publisher, not the world. If a town has no roll call, no gift record and no minutes, that is almost always what the government chose to publish. Say "not published", not "none".
5. Read the quote before you quote the number. Where a figure was extracted from a document, the supporting sentence is stored in the row next to it. If the sentence does not say what the number claims, trust the sentence.

What is machine-extracted and what is not

Two different kinds of record live here and the difference matters for how you attribute them:

• Primary. The meeting existed, the document is at that URL, the payment line is in the government's own file, the filing was filed. This is most of the corpus.
• Derived. Plain-English briefs, extracted decisions, categorised actions. These are produced by language models reading the primary document. They are good enough to find things with and not good enough to quote as fact without opening the source. Every such row keeps a link to the document it was read from — use it.

The practical rule reporters have found useful: search the derived layer, cite the primary one.

Getting bulk data instead

If you want to load this into your own tools rather than click around, do not scrape the portal. The whole corpus is published as a download — Parquet and SQLite on HuggingFace, CC BY 4.0. That copy is built for bulk; this one is built for looking.

When to just email me

Genuinely, this is not a courtesy line. If you are on deadline, or your question spans several tables, or you want to be sure a number is defensible before it goes in print, write to [email protected]. I would much rather spend twenty minutes writing the query correctly than read a figure from this corpus that is wrong.

Attribution, if you use it: MyTown, theboringparts.com, CC BY 4.0. A link is plenty.