Skip to main content

Command Palette

Search for a command to run...

Three Cloud Access-Control Limits That Might Surprise You When Migrating Services at Scale

Updated
5 min readView as Markdown
Three Cloud Access-Control Limits That Might Surprise You When Migrating Services at Scale
M
I'm a software engineer with over 15 years of experience. While my background is primarily in backend engineering, my work today spans cloud infrastructure, AI integrations, and the systems that support modern software applications. I started this blog to share what I’ve learned in a simplified, approachable way — and to add value for fellow developers. Though I’m an introvert, I’ve chosen to put myself out there to encourage more women to explore and thrive in tech. I believe that by sharing what we know, we learn twice as much — that’s precisely why I’m here.

The principle of least privilege is an access-control approach in which a user or service receives only the permissions needed to perform its job.

For example, a service that reads files from a specific group of storage buckets can be given access to only those buckets instead of the entire project. If its credentials are exposed or the service is compromised, the potential impact is limited to a smaller set of resources.

This makes least privilege useful for reducing the scope of security incidents and preventing accidental access or changes to unrelated resources.

In practice, implementing it often involves creating separate identities and narrowly scoped permissions. A service account might access only buckets with a specific prefix. A database user might access only one schema or a subset of tables. A Kafka producer might write only to the topics it uses.

These are just a few examples of least privilege in practice.

During a large cloud migration, these identities and permissions may need to be created for dozens or hundreds of services. That is when platform limits that rarely matter at a smaller scale can become a problem.

Some limits can be increased. Others depend on the pricing tier. And some require a different architecture.

Here are three examples of access-control limits you might encounter during a large-scale migration—and some options for working around them.

1. GCP’s 20 Conditional IAM Bindings Limit

Google Cloud IAM Conditions let you restrict a principal’s access based on attributes such as a resource’s name or type.

Imagine a migration service account that needs to read only from Cloud Storage buckets with a particular prefix. A conditional binding can grant it the Storage Object Viewer role only when the bucket name matches that prefix.

This is a conditional binding in action: the principal and role remain the same, but a condition limits where the access applies.

However, GCP allows only 20 bindings in the same allow policy when they have:

  • The same principal

  • The same role

  • Different conditions

This is a fixed limit. It cannot be increased.

If the same service account and role require separate conditions for different groups of resources, the first 20 bindings may work, but adding the 21st can fail.

Once that happens, the permission model must change. One option is to grant the role directly at the resource level instead of continuing to add conditional bindings to the project-level policy.

A permission model that works well for a few groups of resources may not work at a larger scale.

Google Cloud: IAM quotas and limits

2. MongoDB Atlas’s 100 Database Users Limit

Creating a separate MongoDB Atlas user for each service gives every service its own credentials and permissions.

For example, an orders service may have read and write access to order-related collections, while a reporting service receives read-only access to a smaller subset of data. Their credentials can also be rotated or revoked independently.

MongoDB Atlas allows up to 100 database users per project.

Imagine migrating 80 services and creating one database user for each. That already uses 80 of the 100 available users.

Now add developer accounts, CI/CD identities, operational tools, legacy integrations, and temporary migration jobs. The remaining 20 users can disappear quickly.

MongoDB recommends contacting Atlas Support when more than 100 users are needed. However, its public documentation does not guarantee an increase or specify a higher maximum.

Possible first steps include removing unused users and reviewing whether any identities can be consolidated without weakening the intended access controls. If that is not enough, contact Atlas Support early to understand what options are available beyond 100 users.

MongoDB Atlas: Configure Database Users

3. Confluent Cloud’s Kafka RBAC Role Binding Quotas

Confluent Cloud uses RBAC role bindings to control which Kafka resources a principal can access.

Imagine an order-processing service that:

  • Consumes from the orders-created topic

  • Uses the order-processing consumer group

  • Publishes to the orders-processed topic

Its service account may need separate role bindings for each type of access. This allows the service to consume and publish the messages it needs without granting access to unrelated topics or consumer groups.

The number of cluster-level RBAC role bindings with Kafka permissions depends on the cluster type:

  • Basic, Standard, Enterprise, and Freight: 500 per cluster

  • Dedicated: 25,000 per cluster

If a service needs an average of three bindings, 170 services in the same Kafka cluster could require around 510—more than the 500-binding limit available on several cluster types.

The exact number will vary, but the multiplication is easy to miss when permissions are created one service at a time.

Possible options include consolidating bindings through shared resource prefixes where appropriate, requesting a quota increase if one is available, or moving to a Dedicated cluster. The last option can significantly affect cost.

Access-control requirements should therefore be considered when choosing a cluster, not only throughput and storage.

Confluent Cloud: Service quotas

Final Thoughts

Least privilege is worth aiming for. By limiting what each identity can access, it can improve the security of your services and production environments. However, the identities and permissions used to implement this it are not unlimited.

Before migrating services at scale, count the services and resources that will need access. Estimate how many users, identities, and role bindings that access model will require.

Then design a model that remains granular without exceeding the platform’s limits. If the numbers do not fit, determine early whether the limit can be increased, requires a higher tier, or calls for a different design.

Finding a limit early gives you options. Finding it mid-migration can mean delays, redesigns, or unexpected costs.