AI Enrichment
Running a Model Against Tens of Thousands of Jobs Without Hitting a Wall
Section 3 of 4
Enrichment is where a feed of raw text becomes a usable dataset. Each job gets a standardised title and description, extracted skills and tasks, search terms and tags, an industry classification, a job function within that industry, and a confidence score describing how sure the model is about the classification it just produced.
Doing that once is trivial. Doing it for tens of thousands of jobs per run, against a service with per-region quota limits, where individual calls fail for structurally different reasons requiring structurally different responses, and where the model is sometimes confidently wrong — that is the engineering. This section covers the four mechanisms that make it hold: regional fan-out, self-healing batches, context-correct two-stage classification, and confidence gating.
7
regions in the rotation
800
model calls permitted in flight
10
jobs per model call
3
cache layers before a call is made
2
classification stages
Multi-Region Fan-Out
Quota is granted per region. A single region is therefore a ceiling — and the way past it is not to ask for more, but to use seven of them at once.
The naive design sends everything to one region and moves to another only when the first starts rejecting calls. That is failover, and it means the system spends its time at the edge of the limit of whichever region it is currently exhausting — producing exactly the throttling errors it is trying to avoid, and only discovering the next region is available after failing.
Here, rotation is unconditional. Regions are held in an endless cycle, and every single call takes whichever region comes next. Load is therefore distributed evenly across all seven by construction rather than by reaction, and no individual region approaches its quota until all of them are near it. The available throughput becomes the sum of seven quotas rather than the ceiling of one.
It is also failure isolation as a side effect. A single region degrading affects roughly one seventh of in-flight calls, and those are retried onto different regions by the ordinary retry path, so a regional problem becomes a slowdown rather than an outage.
Self-Healing Batch Splitting
Two failures look similar and are not. A throttling error means try again shortly. A token-limit error means this request will never succeed, no matter how many times you send it — and retrying it is an infinite loop wearing the costume of resilience.
The error handling branches on what the failure actually means. A resource-exhausted error or a timeout is transient — the request is valid and the service is momentarily unwilling — so it is retried with backoff, and because the next attempt takes the next region in the rotation, the retry naturally lands somewhere less loaded.
An invalid-argument error is not transient. In practice it almost always means the batch exceeded the model's token limit, and no amount of retrying will change that. Retrying it burns the retry budget, delays the run, and ends in the same failure. So that branch splits the batch in half and recurses on each half instead.
Splitting works because the failure is a function of size. Half the jobs is roughly half the tokens, and the recursion continues until every sub-batch is small enough to be accepted. Most oversized batches resolve in one or two levels.
The terminal case is the one that makes it safe: a batch of a single job that still fails is not split further and not retried. It is recorded as failed with the specific reason that this job alone exceeds the token limit, and the pipeline moves on. The worst case for one pathological listing with a fifty-thousand-word description is that one listing, not a stalled run.
Unexpected errors get their own branch again — retried with backoff a bounded number of times, then recorded as failed. Nothing loops forever, and nothing fails silently.
Getting Correct Output From the Model
Job function only exists inside an industry. A function that is valid in healthcare is meaningless in construction, and the taxonomy is large enough that presenting all of it at once is both expensive in context and actively harmful to accuracy — the model will happily choose a plausible-sounding function from entirely the wrong branch of the tree.
Classification is therefore split. The first stage assigns industry along with the standardised title, description, skills, tasks and confidence score. Jobs are then grouped by the industry they were just assigned, and the second stage runs per group, presenting only the functions that genuinely belong to that industry.
The grouping is the part that matters. It is what guarantees the model is never shown an option that would be wrong to pick, which makes a whole category of error structurally impossible rather than something to be validated away afterwards. It also shrinks each prompt considerably, since one industry's function list is a small fraction of the full taxonomy.
The industry hierarchy itself is fetched with its own retry handling and cached with a multi-day lifetime, because it is reference data that changes rarely and is needed by every single run.