← Back to Journal

leptons1618@anishgiri.dev:~/journal/pinned-skills-broken-builds-what-happens-when-upstream-changes-without-warning less post.md

#AGENT_WORKBENCH#VERSION_PINNING#DEPENDENCY_MANAGEMENT#PYTHON#SHELL

Pinned Skills, Broken Builds: What Happens When Upstream Changes Without Warning

Sep 7, 2026 · 4 min read

After eight Anthropic skills pinned to exact commit SHAs, one upstream change broke my bootstrap — here's how I caught it and fixed the workflow.

The first time a pinned skill crashed my CI pipeline, I spent forty minutes staring at my own code before I realized the problem was upstream.

I run eight Anthropic skills through the Agent Workbench, each pinned to an exact commit SHA in a manifest file. The bootstrap script validates the manifest, resolves each SHA, and installs globally without touching the upstream repo. It worked reliably for three months until the code-review skill started throwing a traceback on import that it had never thrown before.

The error looked like this:

AttributeError: module 'anthropic' has no attribute 'BedrockPlatform'

The pinned SHA for code-review pointed to a commit from two weeks before the Anthropic SDK shipped a breaking change in their internal module structure. The skill’s requirements.txt pinned anthropic>=0.28, but the new SDK version restructured anthropic.bedrock into anthropic.platform.bedrock, and the import statement in the skill’s review_engine.py hadn’t been updated.

I checked the manifest. The SHA was correct. The install log showed the skill cloned cleanly from the pinned commit. The problem wasn’t my pinning — it was that the skill’s own dependency range was too loose, and a fresh pip install in my CI container pulled the new SDK.

That was the first failure mode I hadn’t anticipated: pinned skill code, unpinned transitive dependencies.

The Immediate Fix

I added a constraints.txt file to the workbench that locks the Anthropic SDK to 0.28.4 across all skill environments. The bootstrap script now installs with:

pip install -c constraints.txt -r skill-requirements.txt

before running any skill code. The constraints file gave me a safety net, but the deeper issue was the workflow — I had no way to know that the upstream skill had a latent compatibility issue until it hit production.

Adding Validation to the Manifest Checker

I added a validation step to the manifest checker that runs each skill’s test suite against the pinned commit in an isolated venv before allowing the pin to be accepted. The script is a forty-line Bash wrapper that:

  1. Clones the skill repo at the specified SHA.
  2. Creates a temporary virtualenv.
  3. Installs the skill’s dependencies from the constraints file.
  4. Runs pytest.

If any test fails, the manifest is rejected and the bootstrap aborts with the failing test output.

The validation step caught a second problem the same day. The sql-gen skill’s test suite passed locally but failed in CI because the CI container was running Python 3.12 while my local machine was on 3.11. The skill used typing.TypeAlias, which was added in 3.12, and the test suite didn’t include a version guard. I added a Python version check to the bootstrap script that reads PYTHON_VERSION from the manifest and compares it against the runtime. If the runtime is older, the install aborts with a clear message:

Skill sql-gen requires Python 3.12+, runtime is 3.11.9

Dependency Conflict Detection

Three weeks in, I noticed another pattern. Two of the eight skills shared a dependency on pandas>=2.0, but one skill needed pandas<2.2 because of a deprecated API call in its data-cleaning module. The constraints file had pandas==2.1.6, which satisfied both, but the manifest didn’t express the relationship. When I updated the constraints to 2.2.0 to fix a security vulnerability in a different skill, the sql-gen skill broke silently — its data-cleaning tests passed because they used a different code path, but the actual cleaning function threw a FutureWarning that got swallowed by the skill’s logging handler.

I added a dependency conflict check to the manifest validator. Before installing, the script builds a temporary requirements file from all eight skills and runs pip check in a dry-run venv. If there are conflicts, it prints the conflicting packages and the skills that require them, then exits with code 1. The output looks like:

CONFLICT: pandas 2.2.0
  sql-gen requires pandas<2.2 (pinned at a3f81c2)
  data-clean requires pandas>=2.0 (pinned at 7b2d4e1)

That single check has prevented three bad deploys in the last month.

The New Lifecycle View

The workbench now handles version pinning as a full lifecycle, not just an install-time concern. The manifest file tracks the skill SHA, the required Python version, and an optional constraints override per skill. The bootstrap script validates the manifest, checks for dependency conflicts, runs the test suite, and installs with constrained dependencies. The CI pipeline calls the bootstrap script on every push to main, and the deploy hook runs the same validation before promoting to staging.

Why This Matters

The original case study described the architecture: manifests, SHAs, bootstrap scripts, no modification of upstream sources. What it didn’t cover is what happens after the initial setup — the upstream changes you don’t control, the transitive dependencies that drift, the silent failures that pass your test suite but break in production. Pinning the commit SHA solves the deterministic bootstrap problem, but it doesn’t solve the compatibility problem. You need the validation layer on top of it, and you need to test against the actual runtime environment, not just your local machine.

The code-review skill is still pinned to the same SHA it was three months ago. The upstream repository has had fourteen commits since then, including two that fixed the Bedrock import path. I haven’t updated the pin yet because I haven’t had a reason to — the constraints file keeps the old SDK version working, and the skill’s tests pass. When I do update, I’ll run the full validation suite, check for dependency conflicts, and promote through staging before production. The pin gives me the choice of when to upgrade, not the obligation to upgrade immediately. That’s the part the architecture alone doesn’t give you.

If you’re pinning skills or any external code, think about the hidden dependencies and runtime mismatches early. The validation step, version checks, and conflict detection turned what could have been a production fire-sale into a few lines of Bash and a constraints file. It’s the little guardrails that keep the whole system from unraveling when upstream decides to refactor.