Did an AI model's risk intuitions match 35 years of federal lending data?

The U.S. Small Business Administration publishes every 7(a) and 504 loan it has guaranteed since fiscal year 1991 — 2190504 rows across 6 files. Before any of it was aggregated, an AI model was handed a list of 20 industry names and 56 state names, with no numbers in it at all, and asked which were riskier. Then the numbers were computed and its answers were graded by code. It placed 21.62% of them in the exactly-right tier, against 20% for guessing.

The grade

• 21.62% of blind predictions in the exact tier
• 20% what guessing scores
• 58 predictions in the wrong tier, of 74 graded
• 5/20 sectors exactly right, 15 wrong (80% within one tier)
• 11/54 states exactly right, 43 wrong (59.26% within one tier)

2 of those misses are as wrong as the scale allows — predicted at one end and observed at the other. They are in the table below, not removed from it.

Rank correlation between what it predicted and what the data says: 0.43 across sectors, 0.18 across states. A correlation and a hit rate answer different questions — a model can order a list well while landing in few exact tiers, or the reverse.

Pick a sector and a state

Every figure below is computed from the federal data. The model's prediction for the sector was made before any of it existed.

What the model said, sector by sector

Ranked by the observed charge-off rate, lowest first. Charge-off rate = charged-off loans ÷ resolved loans, where resolved means paid in full or charged off. Canceled, undisbursed and FOIA-exempt loans are excluded from both sides.

The ingest, replayed

Replay of a recorded ingest run: 2190504 rows from 6 files, read in 13.873658826 seconds. These are the throughput samples the run recorded, played back — not a simulation and not a live run.

What the pipeline threw away, and why

1964709 rows of 2190504 were aggregated. Nothing was dropped silently: every rejected row is counted under a named reason, and a reason that never fired is printed as zero rather than left out.

Reason	Rows	Share of all rows
blank-naics	225761	10.31%
state-not-recognized	28	0%
blank-state	5	0%
gross-approval-not-positive	1	0%
gross-approval-unparseable	0	0%
loan-status-unrecognized	0	0%
naics-sector-unmapped	0	0%
naics-too-short	0	0%
row-has-too-few-fields	0	0%

The largest reason by far is a blank industry code. NAICS did not exist when the oldest loans here were approved, so more than half the 7(a) rows from the nineteen-nineties carry no sector at all. Those rows are counted, named and left out of the sector figures rather than folded into one.

Lenders on the record

Institutions with at least 50 resolved loans, ordered by loans on record. The name is the lender the loan is currently assigned to, which is not always the one that made it.

How this was put together

The predictions and the sector notes come from a real model run. The model saw no data before predicting: its prompt was a list of names. The predictions come from claude-haiku-4-5-20251001 on 2026-08-24. The sector notes come from claude-haiku-4-5-20251001 on 2026-08-24, written after the numbers existed and given them.

Source

U.S. Small Business Administration, retrieved 2026-08-23, data as of 2026-06-30. The dataset on data.sba.gov.

Files

6 CSV files, checked against recorded sha256 digests before a row was read. The raw files are not part of this repository.

Aggregates digest

e00fd47864694e68ae1dca93d14f094b1c1bf3eda02291c04e0d1ea88ee04fe5

Predictions digest

66abe9a560d61cf2b60b285a25fd208e3d3932ed8812dfbc54e97c76143f2690

The prompt the model actually got

This is the whole of what it saw before predicting the sectors. It contains no numerals.

Below is a list of every industry sector in a lending dataset.

Using only your own judgment about American small-business lending, sort each one into exactly one of these five risk tiers, by how likely you think a government-guaranteed small-business loan in it is to be charged off rather than paid in full:

  - much-lower-risk
  - lower-risk
  - typical-risk
  - higher-risk
  - much-higher-risk

Use every tier at least once. Spread your answers across the tiers rather than putting most of the list in the middle one.

The list:
  - Accommodation and Food Services
  - Administrative and Support and Waste Management and Remediation Services
  - Agriculture, Forestry, Fishing and Hunting
  - Arts, Entertainment, and Recreation
  - Construction
  - Educational Services
  - Finance and Insurance
  - Health Care and Social Assistance
  - Information
  - Management of Companies and Enterprises
  - Manufacturing
  - Mining, Quarrying, and Oil and Gas Extraction
  - Other Services (except Public Administration)
  - Professional, Scientific, and Technical Services
  - Public Administration
  - Real Estate and Rental and Leasing
  - Retail Trade
  - Transportation and Warehousing
  - Utilities
  - Wholesale Trade

Reply with a single JSON object and nothing else, of the form:
{"assignments":[{"name":"<exactly as written above>","tier":"<one of the tiers above>","why":"<one short clause>"}]}

Include every name from the list exactly once, spelled exactly as it appears above.