Building an Internal Developer Portal with Backstage, AKS, Crossplane, and Argo CD

Introduction

Modern cloud-native organizations are increasingly adopting Internal Developer Portals (IDPs) to streamline developer experience and platform operations. An IDP serves as a centralized “single pane of glass” for developers to interact with infrastructure and tools, reducing cognitive load while enabling self-service​. Instead of navigating scattered scripts, manuals, and ticketing systems, engineers get a one-stop hub to create and manage resources. This paved road approach to software development saves time and improves developer satisfaction​ by standardizing workflows and automating repetitive tasks.

In this post, we’ll dive deep into why IDPs are critical and how to build one using Azure Kubernetes Service (AKS) as the backbone, Crossplane for infrastructure provisioning, Backstage as the UI portal, and Argo CD for GitOps automation. We’ll examine the role and benefits of each component and demonstrate a working example that empowers developers with self-service while preserving control and governance for SREs and platform engineers.

Why Internal Developer Portals Matter

Traditional devops setups often involve manual hand-offs and custom scripts for provisioning environments, which can lead to slow onboarding, inconsistent environments, and a high volume of support tickets. Internal Developer Portals address these pain points by providing:

• Self-Service Capabilities: Developers can autonomously spin up resources (databases, environments, CI/CD pipelines, etc.) through a friendly interface without filing tickets. This accelerates development cycles and experimentation.
• Standardization and Best Practices: IDPs enforce golden paths – pre-approved templates and configurations – so that every new service or environment adheres to organizational standards (security, naming, monitoring, etc.). This consistency reduces errors and spreads best practices across teams​.
• Reduced Cognitive Load: Rather than learning the intricacies of cloud APIs or YAML configs for every tool, developers use a simplified portal. The IDP abstracts away complexity (via custom APIs and templates) so engineers can focus on coding instead of infrastructure plumbing​.
• Faster Onboarding: New engineers ramp up quickly by using the portal’s catalog of resources and knowledge. They don’t need tribal knowledge to find how to deploy a service or request infrastructure – it’s all documented and accessible in one place​.
• Improved Governance: From an SRE/platform team perspective, all actions funnel through the IDP, which means they can bake in security policies, access controls, and auditing. Compliance is easier since every resource goes through controlled pipelines instead of ad-hoc scripts.

In short, IDPs have become an essential ingredient of platform engineering in cloud-native organizations: they boost developer productivity and autonomy, while enabling platform teams to maintain control and reliability. Now, let’s explore the core components that make such an IDP possible.

Core Components of a Kubernetes-Based IDP

Our IDP will leverage four primary components, each playing a distinct role in enabling self-service on a solid foundation:

Azure Kubernetes Service (AKS) – Orchestration Platform

At the base of our platform is Azure Kubernetes Service, a managed Kubernetes offering. AKS provides the orchestration layer that runs our IDP services (like Backstage, Argo CD, and Crossplane controllers) and hosts any user-provisioned workloads. Using a managed Kubernetes service means we offload master node management to Azure, ensuring high availability and easy scaling without manual effort. AKS also integrates well with Azure identity and networking features, which we can leverage for secure access to cloud resources.

Benefits: AKS delivers a production-ready Kubernetes cluster out-of-the-box, with automatic upgrades and patching. This allows platform engineers to focus on building the IDP functionality rather than managing cluster internals. It provides a consistent runtime for both the platform and the developers’ applications, creating a unified environment.

Provisioning AKS with OpenTofu: In our scenario, the platform engineering team can provision the AKS cluster using OpenTofu for reproducibility and IaC best practices. For example, a simplified OpenTofu configuration might specify the cluster and node pool:

# Configure the Azure provider
provider "azurerm" {
  features {}
}

# Create an AKS cluster
resource "azurerm_kubernetes_cluster" "primary" {
  name                = var.cluster_name
  location            = var.region
  resource_group_name = azurerm_resource_group.main.name
  dns_prefix          = var.dns_prefix
  kubernetes_version  = var.k8s_version

  default_node_pool {
    name                 = "nodepool"
    vm_size              = var.node_vm_size
    node_count           = var.initial_node_count
    enable_auto_scaling  = true
    min_count            = var.min_node_count
    max_count            = var.max_node_count
  }

  identity {
    type = "SystemAssigned"  # enable managed identity for the cluster
  }
}

In the above snippet, we define an AKS cluster with auto-scaling node pools and a system-assigned identity. The identity will later allow Crossplane to interact with Azure on our behalf. After writing the OpenTofu config (including Azure resource group and any needed networking), we run tofu apply to create the cluster. Once AKS is up, we can obtain its kubeconfig and proceed to deploy the remaining components onto it.

Crossplane – Infrastructure Abstraction & Provisioning

With our Kubernetes cluster ready, we turn it into a universal control plane using Crossplane. Crossplane is a CNCF project that extends Kubernetes so it can declaratively manage cloud infrastructure resources (databases, clusters, buckets, you name it) via Kubernetes APIs​. In essence, Crossplane lets us represent infrastructure as custom Kubernetes resources.

Role in IDP: Crossplane is the engine that will fulfill developers’ infrastructure requests. Platform engineers define Composite Resources (XRDs) that model higher-level concepts (like “Environment”, “DatabaseInstance”, “Cache”) combining one or more underlying cloud resources. Developers can then self-service these via simple custom objects, without needing to know the details of OpenTofu or Azure APIs. Crossplane controllers continuously reconcile these objects, creating or updating cloud resources to match the declared state.

Benefits: Crossplane brings declarative GitOps to infrastructure. Because all infrastructure declarations are Kubernetes manifests, they can be stored in Git and managed just like application manifests. Unlike static IaC tools, Crossplane is always running and will notice drift or manual changes, ensuring the actual cloud state matches the desired state. It also integrates with Kubernetes RBAC, so you can restrict who can create certain resources. Most importantly, Crossplane allows infrastructure abstraction – the platform team can expose a simplified custom API for developers (hiding the complex bits). This encapsulation means developers “self-service without needing to become an infrastructure expert.”​

Defining a Composite Resource (XRD) and Composition: Let’s illustrate how platform engineers can create an abstraction. Suppose we want to let developers create a “team environment” with a dedicated database. We can define a CompositeResourceDefinition (XRD) for a TeamEnvironment:

apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: teamenvironments.platform.example.com
spec:
  group: platform.example.com
  names:
    kind: TeamEnvironment
    plural: teamenvironments
  claimNames:
    kind: TeamEnvironmentClaim
    plural: teamenvironmentclaims
  scope: Namespaced
  versions:
    - name: v1alpha1
      served: true
      referenceable: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                dbName:
                  type: string
                dbUser:
                  type: string

This XRD declares a new custom kind TeamEnvironment with some spec fields (e.g. database name and user). Next, we create a Crossplane Composition that maps this high-level resource to actual Azure services:

apiVersion: crossplane.io/v1
kind: Composition
metadata:
  name: teamenv-azure
spec:
  compositeTypeRef:
    apiVersion: platform.example.com/v1alpha1
    kind: TeamEnvironment
  resources:
    - name: sql-db
      base:
        apiVersion: database.azure.crossplane.io/v1alpha1
        kind: PostgreSQLServer
        spec:
          forProvider:
            administratorLogin: ${DB_USER}
            administratorLoginPassword: "auto-generated-pw"
            location: West Europe
            version: "11"
          writeConnectionSecretToRef:
            name: db-conn-${DB_NAME}
      patches:
        - fromFieldPath: "spec.dbName"
          toFieldPath: "metadata.name"
        - fromFieldPath: "spec.dbUser"
          toFieldPath: "spec.forProvider.administratorLogin"

In this composition, whenever a TeamEnvironment is created, Crossplane will create an Azure PostgreSQL server (through the Azure Crossplane Provider) using the provided name and user. The credentials are stored as a Kubernetes Secret (db-conn-name). We could extend this composition with more resources (e.g., a Storage Account, a DNS Zone, etc.) as part of the environment. The key idea is that the developer doesn’t need to know these details – they will simply create a TeamEnvironment object.

After applying the XRD and Composition to the cluster, our platform now offers a new internal API. A developer can request a new environment by submitting a TeamEnvironment YAML (or via the portal UI, as we’ll see). Crossplane will then provision all the cloud resources behind the scenes. This design encapsulates complexity and provides safe defaults and guardrails (e.g., using specific Azure regions or sizing) that the platform team controls.

Backstage – Developer Portal and Self-Service UI

Now we have both our cluster and the infrastructure-as-code engine (Crossplane) in place. Backstage ties everything together as the Internal Developer Portal’s UI. Backstage, originally open-sourced by Spotify, is a web UI platform for developers to discover services, documentation, and perform self-service actions. In our IDP, Backstage will be the front door that developers use to interact with the platform – creating new applications or requesting infrastructure through a catalog of templates.

Benefits: Backstage provides a unified interface that “unifies infrastructure tooling, services, and documentation to create a better and more efficient developer experience”​. It centralizes the Software Catalog (all your microservices and resources), making it easy to navigate what exists. Backstage’s plugin architecture allows integrating various tools (CI/CD, monitoring, cloud provider dashboards, etc.) into one UI. Importantly for IDP, Backstage includes a Software Templates (Scaffolder) plugin, which we will use to offer on-demand creation of resources. By using Backstage, we ensure developers have a single portal to do everything from provisioning a dev environment to checking their service’s build status, which greatly simplifies workflows and accelerates delivery​.

Role in IDP: We will use Backstage’s Scaffolder templates to create a user-friendly form for developers to request resources like the TeamEnvironment composite we defined. When a developer fills out a template (e.g., inputs the database name), Backstage can generate the corresponding YAML and commit it to a Git repository, which then triggers Argo CD to apply it. Backstage will also serve as a catalog for all created resources. For example, once an environment or service is created, it can appear in Backstage’s catalog (via metadata or plugins), giving developers visibility into what they own and its status.

Example Backstage Scaffolder Template: Below is a simplified template definition that could be used to let developers create a new team environment (with a database) by providing just a few inputs. This template will ultimately produce a YAML manifest for a TeamEnvironmentClaim (the claim is a namespaced version of our composite resource):

apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
  name: create-team-environment
  title: Create Team Environment
  description: Provision a new team environment with an Azure DB via Crossplane.
spec:
  owner: platform-team
  type: service

  parameters:
    - title: Environment Details
      required:
        - envName
        - dbUser
      properties:
        envName:
          type: string
          description: Unique name for your environment (also DB name).
        dbUser:
          type: string
          description: Admin username for the database.

  steps:
    - id: publish-yaml
      name: Commit Environment Definition
      action: publish:github
      input:
        repoUrl: github.com:my-org/platform-configs
        filePath: environments/$-env.yaml
        content: |
          apiVersion: platform.example.com/v1alpha1
          kind: TeamEnvironmentClaim
          metadata:
            name: $
          spec:
            compositionSelector:
              matchLabels:
                env: "azure-dev"
            resourceRef:
              apiVersion: platform.example.com/v1alpha1
              kind: TeamEnvironment
              name: $-xr
            params:
              dbName: $
              dbUser: $
        commitMessage: "Provision new team environment $"

This Backstage template presents a form asking for an environment name and DB username. When run, it will generate a YAML file (using the input values) and commit it to a Git repository (platform-configs). The content in this case is a TeamEnvironmentClaim custom resource referring to our Crossplane composite. We’re using the publish:github built-in action to push the file to a repo, but we could also run kubectl apply directly in a step. Committing to Git is a nice approach because it lets GitOps (Argo CD) take over from here.

With this template in Backstage’s catalog, a developer just clicks “Create Team Environment”, fills in the fields, and Backstage handles the heavy lifting. In the background, the YAML is generated and stored in Git – ready for Argo CD to deploy.

Argo CD – GitOps Continuous Delivery Engine

Last but not least, Argo CD serves as our GitOps controller to ensure everything flows automatically from Git to the cluster. Argo CD watches the Git repository (or repositories) where platform configurations and application manifests reside. In our IDP setup, Argo CD has two important duties:

1. Deploying and Configuring Platform Components: The platform team can use Argo CD to manage the installation of Crossplane, its providers, and even Backstage itself on the AKS cluster. By keeping these deployment manifests in Git, the platform’s state is reproducible and version-controlled. (For example, a Helm release for Backstage or Crossplane Helm chart can be synced by Argo CD.)

2. Applying Developer Self-Service Changes: When a Backstage scaffolder template commits a new resource definition (like our TeamEnvironmentClaim YAML), Argo CD will detect the change in the Git repo and apply that manifest to the cluster. This triggers Crossplane to provision the requested Azure resources. Argo CD essentially glues Backstage to Crossplane in a GitOps fashion – developers’ intents get recorded in Git and Argo ensures the cluster reflects that state.

Benefits: Argo CD provides continuous reconciliation of declared state. It guarantees that the actual cluster state (Kubernetes + cloud resources via Crossplane) matches the source of truth in Git. This brings auditability (every change is in Git history) and easy rollback (revert a git commit to undo a provision). For SREs, Argo’s dashboard also offers visibility into all applied configs and any drift. Using Argo CD, we achieve a fully automated pipeline: a developer’s request in Backstage goes through Git and is realized in the cloud without any manual kubectl or console steps.

We can install Argo CD on AKS via Helm or kubectl; once running, we define an Application manifest that points to our Git repos (e.g., one for platform infra and one for developer-created resources). For example, the platform team might have:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: platform-resources
spec:
  destination:
    namespace: argocd
    server: https://kubernetes.default.svc
  source:
    repoURL: https://github.com/my-org/platform-configs.git
    targetRevision: main
    path: infra/ # folder containing Crossplane XRDs, Compositions, etc.
  project: default
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

And another Application for the dev environments (e.g., path: environments/). With automated sync, Argo will continuously apply changes. If a developer’s commit adds environments/prod-env.yaml, Argo applies it, Crossplane creates an Azure DB, and voila – the developer’s environment is up. Argo CD ensures consistency and greatly reduces the chance of configuration drift or human error in the process.

Benefits of an IDP for Developers, Platform Engineers, and the Organization

Having walked through the architecture, let’s summarize how an Internal Developer Portal built with these tools translates to concrete benefits for different stakeholders:

For Developers – Self-Service, Speed, and Consistency

• Frictionless Self-Service: Developers can obtain infrastructure or scaffold new microservices with a few clicks in Backstage, without waiting on other teams. This on-demand access means they spend less time blocked and more time building features.
• Faster Delivery: Boilerplate setup (CI pipelines, deployments, cloud resources) is automated through templates. A process that might have taken days of coordination is reduced to minutes. This “operate at the speed of software” approach was called the “paved road” by Gartner, as it streamlines development and even improves developer job satisfaction​.
• Consistent Environments: Because all resources are created from standardized templates and compositions, developers enjoy environments that are consistent across dev/test/prod. Fewer “works on my machine” issues arise since the IDP enforces uniformity in how apps and infra are configured.
• Single Unified Portal: Developers don’t need to juggle dozens of tools or URLs. The IDP brings together documentation, service catalogs, CI/CD status, and provisioning in one place. This reduces context switching and cognitive overhead, allowing engineers to focus on coding and innovation.

For SREs / Platform Engineers – Control, Governance, and Scalability

• Policy Enforcement & Guardrails: Platform engineers define what “good” looks like via OpenTofu, Crossplane compositions, and Backstage templates. This ensures every resource provisioned meets security and compliance guidelines (e.g., using approved base images, in approved regions, with proper tagging).
• Centralized Governance: All infrastructure changes flow through a controlled GitOps process. SREs have full visibility into what’s being created and changed. They can set role-based access in Kubernetes (for who can create certain custom resources) and use approvals in Git if needed. This greatly reduces the risk of rogue cloud resources or snowflake setups.
• Scalability of Operations: Automation is a force multiplier. Instead of handling individual requests or tickets for dev resources, platform engineers invest time in improving templates and platform APIs. Onboarding a new team or project becomes a repeatable event (often self-service). SREs can manage more infrastructure with less manual effort because the IDP handles the common tasks.
• Reduction in Support Load: With developers empowered to self-service and clear golden paths defined, the number of “please set up X for me” JIRA tickets or Slack requests declines. The platform team is freed to work on platform improvements rather than toil. They can also proactively monitor platform health (since Argo CD and Crossplane provide status metrics) rather than firefighting ad-hoc changes.

For the Organization – Compliance, Efficiency, and Developer Velocity

• Improved Compliance and Auditability: Every change to infrastructure is tracked in Git and goes through the IDP. This audit trail makes it easier to demonstrate compliance with regulations or internal policies. Changes are transparent and can be reviewed, and unwanted changes can be rolled back by reverting commits.
• Resource Optimization: An IDP often leads to more consistent use of infrastructure, which can highlight inefficiencies. For example, if every project uses a standardized environment, it’s easier to track costs or implement cost-saving measures across all of them. Also, eliminating manual config drift means fewer outages or incidents due to misconfigurations, which has downstream cost benefits.
• Higher Developer Productivity (and Happiness): A well-implemented IDP removes obstacles and frustrations from engineers’ daily work. When developers spend less time figuring out how to request a database or deploy their app, and instead get these done in a self-serve manner, they can deliver business value faster. This directly boosts the organization’s ability to innovate and respond to market needs. Moreover, a smooth developer experience aids in talent retention – developers prefer environments where they can be productive and autonomous.
• Reduced Time-to-Market: By streamlining the path from idea to deployment, an IDP enables the organization to ship features more frequently and reliably. The combination of templates, GitOps, and automated provisioning ensures that once code is ready, everything else (infrastructure, pipelines, approvals) is not a bottleneck. This competitive advantage can be significant in fast-moving industries.

Bringing It All Together: IDP Workflow in Action

To visualize the workflow: a developer accesses Backstage (running on AKS) and uses a template to request a new environment or service. Backstage (via the scaffolder) generates the required manifest (for instance, a TeamEnvironmentClaim custom resource) and commits it to the Git repo. Argo CD, already watching that repo, notices the new file and applies it to the cluster. The Crossplane operator on AKS sees the creation of the TeamEnvironmentClaim and, based on our Composition, provisions the Azure PostgreSQL database and any other defined resources. In a matter of moments, the developer’s requested environment is live on Azure. The connection details (credentials, endpoints) may be stored in secrets which developers can fetch, or even surfaced in Backstage through plugins.

Meanwhile, the Backstage UI could be extended to show the status of the provisioning. For example, using Argo CD’s plugin, developers can see if their “environment deployment” is synced or if any errors occurred. Or using Crossplane’s UI plugin, they could inspect the health of the TeamEnvironment resource from within Backstage. All these integrations further enrich the portal as a true hub of information and action.

Crucially, if a developer no longer needs a resource, they can use the portal to mark it for deletion (which might, for example, remove the YAML from Git or mark the resource as deleted), and the same machinery (Argo and Crossplane) will deprovision it, keeping cloud clutter to a minimum. Everything is declarative and tracked.

Conclusion

Building an Internal Developer Portal with AKS, Crossplane, Backstage, and Argo CD unlocks a powerful synergy of tools: developers gain a self-service interface to cloud infrastructure and dev tools, while platform engineers enforce consistency and automate operations. We end up with a win-win scenario – faster development cycles for engineers, and a scalable, controllable platform for SREs and the organization.

This stack embodies the key principles of modern platform engineering: encapsulation of complexity, GitOps for continuous reconciliation, and developer experience as a first-class priority. By adopting an IDP, organizations can significantly reduce onboarding time, eliminate manual bottlenecks, and achieve greater reliability across the software delivery lifecycle. If you’re looking to boost developer productivity and operational control, consider giving this AKS + Backstage + Crossplane + Argo CD approach a try. It may require an upfront investment in setup and learning, but the payoff in the long run – in developer velocity and platform stability – is well worth it.

With the blueprint and examples provided above, you have a starting point to implement your own Internal Developer Portal. Happy building, and welcome to the world of faster, safer, and happier cloud-native development!