Files
adr-sml/public/robots.txt
T
Pouya LajevardiandClaude Opus 5 17e316dc1d
Build and deploy / build-and-deploy (push) Failing after 4s
feat: park the two policy changes the pricing plan forbids; robots.txt stands in
The third --apply of 2026-09-04 reached update-distribution and was rejected
atomically: "Distributions with the Free pricing plan can't have the following
features: Custom origin request policy, Custom response headers policy."
Pouya's ruling: both are PARKED as unavailable — a platform constraint, not a
defect.

The pre-flight added in the previous commit could not have caught this, and
that is the point: every limit in PAYLOAD_LIMITS is a property of the payload,
while this is a property of the account, reported only by the call the
pre-flight exists to avoid. Both sections now stop before creating anything.

The plan is not in the CloudFront API — checked across 167 operations, no
operation, shape, member or documentation string mentions one, and
PriceClass_All is the edge-location price class, not the plan. So the gate is a
constant, PLAN_ALLOWS_CUSTOM_POLICIES, and the two sections report as PARKED
under their own heading rather than as skips: the previous commit made a skip
exit 3, and a constraint true on every run would have made 3 permanent. Proven
with a shim that refuses every mutating verb: --apply now makes zero of them.

Substitute (a): Disallow: /pouya-lajevardi-bio.pdf in robots.txt, placed before
Allow:/ so first-match crawlers honour it too. It is not an equivalent and the
file says so — it stops the PDF being fetched, solving the duplicate-of-/bio/
problem, but does not de-index a URL linked from /bio/ and /about/. Verified:
syntax, a match simulation under both crawler semantics, and that the sitemap
does not list the PDF.

Substitute (b): the WAF web ACL CreatedByCloudFront-f8fbf256 is already
attached — 925 WCU, three AWS managed rule groups, no rate-based statement.
That corrects §9 Q65, which framed WAF as a cost decision about adding one and
named the now-unappliable header forwarding as its groundwork. The real
question is one rule on an ACL already paid for, and a rate-based rule matches
the viewer address directly, so the capability is superseded rather than lost.

Reviewed in two rounds by me rather than a separate agent, per instruction.

Nothing was applied to the distribution and nothing was deployed; robots.txt
needs one site deploy.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Md3GndFqWPzK78xAoebsg5
2026-09-04 14:52:13 -04:00

32 lines
1.7 KiB
Plaintext

# adr.smlcompany.ca
# AI crawlers are deliberately allowed. Being read by an assistant that counsel
# is using to shortlist a neutral is the point. See docs/04-seo-spec.md.
# EXACTLY ONE THING IS DISALLOWED, AND IT IS A SUBSTITUTE FOR A MECHANISM THIS
# DISTRIBUTION IS NOT ALLOWED TO HAVE.
# The bio PDF duplicates /bio/. The intended fix was `X-Robots-Tag: noindex` on
# *.pdf via a CloudFront response-headers policy, which the distribution's
# pricing plan forbids — AGENTS.md §7, and docs/09 Part 3 for the three failed
# attempts. Disallow is the remaining lever and it is NOT the same instrument:
# - it stops the PDF being FETCHED, so its contents are never indexed and the
# duplicate-content problem it was raised for is solved;
# - it does NOT de-index the URL. The PDF is linked from /bio/ and /about/, so
# a search engine can still list the bare URL with no snippet.
# That residual is accepted deliberately: a bare link to a bio PDF is not the
# harm the noindex was for. Revisit if the pricing plan ever changes.
#
# /legal/* is still NOT disallowed, and that reasoning is unchanged — it is the
# general rule this one path is the exception to. A crawler forbidden to FETCH a
# URL never reads the `noindex` on it, so the two cancel; the legal pages are
# linked from every footer, so they are discovered regardless, and the pair would
# have produced bare "no information available" listings with the directive that
# would have suppressed them sitting unread behind the wall.
# `noindex` is what de-indexes; `Disallow` is what prevents fetching. Use the one
# that matches the problem, and never both on the same path.
User-agent: *
Disallow: /pouya-lajevardi-bio.pdf
Allow: /
Sitemap: https://adr.smlcompany.ca/sitemap-index.xml