Response Header in CloudFront

Problem:

Google is indexing your staging and dev environments, which could result in duplicate content across environments. This indexing can harm the SEO of your production website by placing it in "supplemental results" (lower priority in search rankings). To avoid this, you want to add the `X-Robots-Tag: noindex, nofollow` header to your website hosted on S3 + CloudFront.

Solution:

1. Create a Response Headers Policy in CloudFront:
   • Open your CloudFront distribution in the AWS console.
   • Go to the Behaviors tab and edit the relevant cache behavior (e.g., /* for all content).
   • Add a Response Headers Policy by creating a custom policy that adds the header:
     • X-Robots-Tag: noindex, nofollow
     • Ensure to select Overwrite if the header already exists.
2. Attach the Policy to the Cache Behavior:
   • Apply the newly created Response Headers Policy to the desired cache behavior (e.g., the default /* path).
3. Invalidate the CloudFront Cache:
   • Optionally, create a cache invalidation for /* to ensure that the new header is applied immediately.

Outcome:

This will ensure that your staging and dev environments are no longer indexed by search engines, preserving the SEO ranking of your production site.