SOCI v2 on ECS - what's new

31 August 2025

Around May 2025, AWS started updating SOCI Snapshotter GitHub repo to support SOCI indexes v2. This is a refreshed model that simplifies the index management. It's easier to find which index refers to which image/tag with this new version. Because of that I also updated the tool I have presented in the previous post, so that it also supports latest SOCI version.

Previous post you can find here: https://pabis.eu/blog/2025-06-17-Faster-ECS-Startup-SOCI-Index-GitLab-Pipeline.html

The new standalone SOCI indexer is in this GitHub repo: github.com/ppabis/ecr-aws-soci-index-builder/tree/v0.11.2

Disclaimer ⚠️

SOCI V1 feature is disabled by default in all accounts that did not submit a Support ticket to AWS. As previously mentioned, this wasn't announced anywhere but Support confirmed eventually that SOCI V1 is gated since beginning of 2025. SOCI V2 is enabled by default in all accounts. What is more, very recently AWS announced (August 27, 2025) that SOCI V1 will be deprecated until February 9, 2026.

📦 Architecture for our experiments

If you want to play around with these tools, you can clone the repository from here. I used OpenTofu but it should also work well with Terraform. Just provide the credentials and run tofu apply.

https://github.com/ppabis/soci-v2-experiments

I have created a VPC with all the essentials like NAT gateway, subnets, two EC2 instances that can be used for building the images and indexes, ECR repository and a simple ECS Fargate service. Once you create this environment, wait two or three minutes because the EC2 instances should build the images for respective architectures and push them to ECR on startup. After that you can SSH into one of them using EC2 Instance Connect and create an image index of both images.

# Log in to any instance. Check tofu output to get the commands
aws ec2-instance-connect ssh \
   --region us-east-2 \
   --instance-id i-05234fc9c1275999d \
   --instance-ip 10.60.2.140 \
   --connection-type eice \
   --os-user ec2-user

Now using the following commands, we can create a multi-arch image. This is a good practice if you sometimes want to save on costs - for example it can happen that x86 spot instances are cheaper at this point in time. As the images can be built in parallel, the cost is very small while there is a good potential for savings in the future.

# Change this to your registry and region. Check tofu output to get the value.
export ECR_REGISTRY="889900112233.dkr.ecr.us-east-2.amazonaws.com"
aws ecr get-login-password | docker login -u AWS --password-stdin $ECR_REGISTRY
docker manifest create \
 $ECR_REGISTRY/soci-repo:latest \
 $ECR_REGISTRY/soci-repo:arm64 \
 $ECR_REGISTRY/soci-repo:amd64

docker manifest annotate \
 $ECR_REGISTRY/soci-repo:latest \
 $ECR_REGISTRY/soci-repo:arm64 \
 --arch arm64

docker manifest annotate \
 $ECR_REGISTRY/soci-repo:latest \
 $ECR_REGISTRY/soci-repo:amd64 \
 --arch amd64

docker manifest push $ECR_REGISTRY/soci-repo:latest

In your ECR repository you should now see three images: two actual images and one index like on the image below. These are not SOCI enabled.

We can use ORAS to analyze how this new manifest looks like. It should have just two entries for Intel image and ARM image. On the instances I have already installed ORAS using user data but you can get more information about it here: oras.land.

aws ecr get-login-password | oras login -u AWS --password-stdin $ECR_REGISTRY
oras manifest fetch $ECR_REGISTRY/soci-repo:latest

{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 2613,
         "digest": "sha256:cf8488c22543c8ac676dbea9ceb99abb7120799405b05b4098149bed20bf14b3",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 2613,
         "digest": "sha256:a60af31eb40bb689c25aa23748707fdbd0d698734bbdb58ba3ee3597fb214379",
         "platform": {
            "architecture": "arm64",
            "os": "linux"
         }
      }
   ]
}

Creating SOCI V1 index 1️⃣

We can now create old SOCI V1 index using the tool that should also be available on the instance. You can run it in Docker and just provide the URIs as well as requested SOCI index version. With V1 you can't point to an index as it will result in an error.

$ docker run --rm -it -e AWS_REGION=us-east-2 soci-gen:latest -repository $ECR_REGISTRY/soci-repo:latest -soci-index-version V1
{"level":"warn","RegistryURL":"889900112233.dkr.ecr.us-east-2.amazonaws.com","time":"2025-08-29T07:00:26Z","message":"Image manifest validation error: not a valid image manifest: empty config media type"}
# Generate SOCI V1 indexes for arm64 and amd64 separately
$ docker run --rm -it -e AWS_REGION=us-east-2 soci-gen:latest -repository $ECR_REGISTRY/soci-repo:amd64 -soci-index-version V1
$ docker run --rm -it -e AWS_REGION=us-east-2 soci-gen:latest -repository $ECR_REGISTRY/soci-repo:arm64 -soci-index-version V1

In ECR you will see now two additional entries that are untagged. They will be of type application/vnd.amazon.soci.index.v1+json. However, even if you now download the manifest of any of the images or image index, nothing will change. SOCI V1 index is just a manifest uploaded to the repository.

🤔 The problem with V1 indexes

In order to find if an index exists for a specific image, you either have to scan all the indexes in the repo or have some database mechanism that stored a map of indexes and image hashes. A way to imagine this problem is with simple files and directories. Let's say you have a directory that represents your container image repository. You have several tar.gz files that are actual images and some soci.json files that are SOCI indexes (this example is just simplification).

$ ls myrepo
78160dbe.tar.gz         685511a5.tar.gz         3385ce09.soci.json
632fe5f2.soci.json      5663eace.soci.json

Let's assume that you want to pull image 78160dbe. To check if you have a SOCI index for this image, you need to either check all the .soci.json files or have some database that has a list of that. In each .soci.json file, there's a "Subject" field that has a hash of the image it relates to.

When you download the JSON that sits behind this entry, you can see the SHA sum of the image for which the SOCI index was created (at the bottom under subject). When you look at the JSON manifest of the image, there's no relation to the SOCI index - it's just a plain old Docker manifest.

# Fetch the SOCI V1 index
oras manifest fetch 0123456789012.dkr.ecr.us-east-2.amazonaws.com/soci-repo@sha256:d00c6532ae6925dc10527d85eb77ec048c5661bf8a6bf460ee53be474e38f5de | jq

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "config": {
    "mediaType": "application/vnd.amazon.soci.index.v1+json",
    "digest": "sha256:d00c6532ae6925dc10527d85eb77ec048c5661bf8a6bf460ee53be474e38f5de",
    "size": 2
  },
  "layers": [
    {
      "mediaType": "application/octet-stream",
      "digest": "sha256:5a317781e98d5b9e06ff67dfcd721fde46eb30e48af5ec959ff15260124521d9",
      "size": 1387368,
      "annotations": {
        "com.amazon.soci.image-layer-digest": "sha256:fbe4cc75eed0371e58892a650547fc4b104356b21b2da1ef76c9219247af9cc9",
        "com.amazon.soci.image-layer-mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip"
      }
    },
    ... // More ztoc layers continue
  ],
  "subject": {
    "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
    "digest": "sha256:2b0d259f8f55f97f4052afd5bba0cc54cc394206767de85386a379493a75af8d",
    "size": 2613
  },
  "annotations": {
    "com.amazon.soci.build-tool-identifier": "AWS SOCI CLI v0.2"
  }
}

To find if an image has a SOCI index, we can use oras discover command. It finds the relationships between untagged artifacts 😮. For example to locate the SOCI v1 index of arm64 image, I use the following command. This is this magical database I was talking about. Notice that the hash of the index matches what we just downloaded.

oras discover 0123456789012.dkr.ecr.us-east-2.amazonaws.com/soci-repo:arm64
0123456789012.dkr.ecr.us-east-2.amazonaws.com/soci-repo@sha256:a60af31eb40bb689c25aa23748707fdbd0d698734bbdb58ba3ee3597fb214379
└── application/vnd.amazon.soci.index.v1+json
    └── sha256:d00c6532ae6925dc10527d85eb77ec048c5661bf8a6bf460ee53be474e38f5de

On my current account, where I requested SOCI V1 to be enabled, it still works. But when I tested on another clean account, it indeed did not show up in ECS. The example service in the repository fetches ECS metadata of the task and the container. If the Snapshotter field is soci that means either SOCI V1 or V2 was used. If it is overlayfs that means no SOCI index was found or you are trying to use SOCI V1 on an account where it is disabled.

Upgrade to SOCI V2 🚀

I used a new version of the standalone indexer I modified from the Lambda tool. It does not overwrite the original tag but creates a new one with -soci suffix. That way you can have both SOCI-indexed alongside same non-indexed image in the repository. Back in SOCI v1, unless deleted, the index was always retrieved for a particular image if the runtime has the feature enabled.

With SOCI V2 things are more reliable. The index is also kept in a similar file, however, the connection between index and image is stored both in the index file itself but also in another manifest file that is tagged. So once you pull myimage:latest-soci, a manifest is loaded first that contains links to both the actual layered image as well as the SOCI index. It makes it quicker and less complicated to manage. What is more, you can control whether you want to use SOCI enhanced image or a normal one by changing a tag.

I have built SOCI V2 index against the multi-arch index we created earlier, as this use case is supported now. The command is almost the same. However, you can also point to the image directly. In both cases, a new index will be created.

On the image above you can see also some two untagged images. These are the same as the tagged arm64 and amd64 images but their manifests were converted to OCI format (from a default Docker one). If you inspect the two, the layers are untouched and have the same hashes - only media type is changed. You won't pay for more storage as the layers are shared. I verified that with ORAS.

# Fetch original Docker manifest
oras manifest fetch 0123456789012.dkr.ecr.us-east-2.amazonaws.com/soci-repo:arm64
# Fetch converted OCI manifest of the same image
oras manifest fetch 0123456789012.dkr.ecr.us-east-2.amazonaws.com/soci-repo@sha256:cdd640f4e50444a0be5e10c5698bc27904e68e1246b90acf18521b4117df83da | jq

{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   "config": {
      "mediaType": "application/vnd.docker.container.image.v1+json",
      "size": 8072,
      "digest": "sha256:873cf5dabf17fcf461fbfc55c9785fa2fcf282b55b96e6ef0508a944166cb514"
   },
   "layers": [
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 31373560,
         "digest": "sha256:fbe4cc75eed0371e58892a650547fc4b104356b21b2da1ef76c9219247af9cc9"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 1270307,
         "digest": "sha256:3cd5add3ba3330f959ed1b27096bfb25e2f29a4cf327009d69855a903036f3af"
      },
    ...
   ]
}

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "config": {
    "mediaType": "application/vnd.oci.image.config.v1+json",
    "digest": "sha256:873cf5dabf17fcf461fbfc55c9785fa2fcf282b55b96e6ef0508a944166cb514",
    "size": 8072
  },
  "layers": [
    {
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "digest": "sha256:fbe4cc75eed0371e58892a650547fc4b104356b21b2da1ef76c9219247af9cc9",
      "size": 31373560
    },
    {
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "digest": "sha256:3cd5add3ba3330f959ed1b27096bfb25e2f29a4cf327009d69855a903036f3af",
      "size": 1270307
    },
  ...
  ]
}

☹️ Multi-arch limitation?

As least with the tool I have converted from Lambda there actually is a limitation. Even though as you saw, both arm64 and amd64 images were converted to OCI format, SOCI index was only created for the platform you run the tool from. It essentially behaves the same was as you would pull a default image. When I tried to force creation of amd64 index on arm64 instance, it failed by trying to download actual arm64 image from amd64 tag. When you fetch the manifest of the -soci tagged image, you can see that it only contains one SOCI entry and two OCI image entries for both architectures.

# Get new image index manifest generated by SOCI v2
oras manifest fetch 0123456789012.dkr.ecr.us-east-2.amazonaws.com/soci-repo:latest-soci

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:6771831a7bfa83d50a462058c462c4b5b7ebc5ad47bc72bc9894f85bfedbbeef",
      "size": 2027,
      "platform": {
        "architecture": "amd64",
        "os": "linux"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:cdd640f4e50444a0be5e10c5698bc27904e68e1246b90acf18521b4117df83da",
      "size": 2148,
      "annotations": {
        "com.amazon.soci.index-digest": "sha256:963e2cc55b42e73b6f51530d9956ed03cb204bf0fce0b2608a1a4e7674e7c354"
      },
      "platform": {
        "architecture": "arm64",
        "os": "linux"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:963e2cc55b42e73b6f51530d9956ed03cb204bf0fce0b2608a1a4e7674e7c354",
      "size": 1394,
      "annotations": {
        "com.amazon.soci.image-manifest-digest": "sha256:cdd640f4e50444a0be5e10c5698bc27904e68e1246b90acf18521b4117df83da"
      },
      "platform": {
        "architecture": "arm64",
        "os": "linux",
        "variant": "v8"
      },
      "artifactType": "application/vnd.amazon.soci.index.v2+json"
    }
  ]
}

However, when I created a new manifest and merged two SOCI V2 image indexes together (4 entries in total) along with all annotations to match, the ECS service as able to use SOCI snapshotter for both architectures so the limitation is only in the tool. You can find the example index here: manifest-new.json. I used ORAS to forcefully push it to ECR.

oras manifest push 889900112233.dkr.ecr.us-east-2.amazonaws.com/soci-repo:latest-soci-2 manifest-new.json

But! I asked in the issues of the original SOCI Snapshotter for the feature and it turns out it was there under the flag --all-platforms. By analyzing the code I backported it to my tool and now it generated all the platforms by default (you can't change this behavior).

Conclusion

SOCI V2 is not a big or breaking change. It simplifies the management of indexes and makes it easier for the registry provider to handle them. They are treated the same way as regular artifacts stored in the registry and always bound by a manifest.