I’ve been using Azurite to simulate Azure storage for my development. If you’re not familiar with it, Azurite is a local storage emulator for Azure Storage, and you can read my other post about how i’ve set up my devcontainer configuration to run Azurite as a service container. As my deployed code is using an Azure Managed Identity, I wanted ensure my development environment was consistent with this and also uses Azure DefaultAzureCredential credential provider class. In this post, i will talk through the steps required to switch from using a connection string (with a well-known account and key) to using OAuth and HTTPS, helping to increase feature parity between development and production, reducing the chances of mistakes.

There are essentially 5 steps:

Obviously, this is only useful for development, and you shouldn’t use this to secure services running directly on the internet.

Create a local Certificate Authority

The first hurdle is to set up a CA, and issue a certificate for Azurite to use. By far the simplest way is to use minica – a simple CA which generates a root certificate and any number of other certificates. The other tool i found is mkcert but i didn’t try it.

We could set this up so that it’s built every time we rebuild the devcontainer, but the minica certificate is valid for over 2 years, so it’s probably not worth it, so instead just install minica on your local and generate the certificates which we can then copy to the repo. There are installation instructions on the minica repo – I did this on my Mac:

~/Downloads > brew install minica
==> Downloading https://ghcr.io/v2/homebrew/core/minica/manifests/1.1.0
Already downloaded: /Users/rob/Library/Caches/Homebrew/downloads/291ff83573a0a9e0a7033accd18d58fcf701211c2b7b63a31e49c36fabc0cb5f--minica-1.1.0.bottle_manifest.json
==> Fetching minica
==> Downloading https://ghcr.io/v2/homebrew/core/minica/blobs/sha256:dc8955ffd5c34b8eaedbc556e71188ec55c2a01e76c26f853aeb0038c7ac2426
############################################################################################################################################################################################################### 100.0%
==> Pouring minica--1.1.0.arm64_sonoma.bottle.tar.gz
🍺  /opt/homebrew/Cellar/minica/1.1.0: 6 files, 4.3MB
==> Running `brew cleanup minica`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
~/Downloads > mkdir azurite-certs
~/Downloads > cd azurite-certs
~/Downloads/azurite-certs > minica -ip-addresses 127.0.0.1
~/Downloads/azurite-certs > tree
.
├── 127.0.0.1
│   ├── cert.pem
│   └── key.pem
├── minica-key.pem
└── minica.pem

2 directories, 4 files

The minica-key.pem and minica.pem files are the CA’s private and public keys respectively. The 127.0.0.1 folder contains the private key and certificate for the hostname 127.0.0.1.

Be sure to use the argument -ip-addresses and not -domains – Node requires that IP addresses are present in the Subject Alternative Name (SAN) field of certificates. If you accidentially use the -domains option, you’ll get an ERR_TLS_CERT_ALTNAME_INVALID error when you try to connect from Azure Storage Explorer.

Examining the certificate, it looks like this:

~/Downloads/azurite-certs/127.0.0.1 > openssl x509 -in cert.pem -text -noout
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 4677514164283179045 (0x40e9de5991f50025)
        Signature Algorithm: ecdsa-with-SHA384
        Issuer: CN=minica root ca 190f9e
        Validity
            Not Before: Oct  6 10:03:35 2024 GMT
            Not After : Nov  5 11:03:35 2026 GMT
        Subject: CN=127.0.0.1
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (384 bit)
                pub:
                    04:...:eb
                ASN1 OID: secp384r1
                NIST CURVE: P-384
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Authority Key Identifier: 
                F1:D0:94:63:AA:37:F6:EF:CF:5F:CD:83:80:2C:95:D0:76:6C:2A:07
            X509v3 Subject Alternative Name: 
                IP Address:127.0.0.1
    Signature Algorithm: ecdsa-with-SHA384
    Signature Value:
        30:...:50

I copied the entire azurite-certs folder to the .devcontainer folder of my project, and renamed the folder 127.0.0.1 to certs (as i found i couldn’t mount the originally named folder into the container).

Configure Azurite to use the certificate (and enable OAuth with basic checking)

This is relatively easy. Azurite only really needs to be told the path to the certificate and private key from the 127.0.0.1 folder. To do this, we can mount the folder in to the container, and pass the path to Azurite in the command:

services:
  devcontainer:
    ...

  azurite:
    image: mcr.microsoft.com/azure-storage/azurite
    ports:
      - "127.0.0.1:10000:10000"
      - "127.0.0.1:10001:10001"
      - "127.0.0.1:10002:10002"
    command: >
      azurite
      --blobHost 0.0.0.0
      --queueHost 0.0.0.0
      --tableHost 0.0.0.0
      --cert /workspace/certs/cert.pem
      --key /workspace/certs/key.pem
      --oauth basic
    volumes:
      - ./azurite-certs/certs:/workspace/certs

Configure the devcontainer to trust the certificates

This step is more complicated. To enable trust, you need to install the minica root certificate in to the relevant trust stores inside the container. Thankfully, i found this script which does the trick. To use it, we’ll create our own Dockerfile which defines the devcontainer. We’ll base it on the existing image, and add a couple of steps

FROM mcr.microsoft.com/devcontainers/python:1-3.11-bullseye

# Switch to root user to install packages and update certificates
USER root

# Install ca-certificates package and libnss3-tools
RUN apt-get update && apt-get install -y ca-certificates libnss3-tools

# Copy the minica certificate to the container and install it
COPY ./azurite-certs/minica.pem /usr/local/share/ca-certificates/minica.crt
COPY ./azurite-certs/trust_minica.sh /usr/local/bin/trust_minica.sh

# Update CA certificates
RUN chmod +x /usr/local/bin/trust_minica.sh
RUN /usr/local/bin/trust_minica.sh
RUN update-ca-certificates

# Switch back to the non-root user (devcontainer default user)
USER vscode

# Keep the container running
CMD ["/bin/sh", "-c", "while sleep 1000; do :; done"]

And we’ll put the script (not my work – from this Gist) in the azurite-certs folder as trust_minica.sh:

#!/bin/sh

### Script installs minica.pem to certificate trust store of applications using NSS
### https://gist.github.com/mwidmann/115c2a7059dcce300b61f625d887e5dc
### (e.g. Firefox, Thunderbird, Chromium)
### Mozilla uses cert8, Chromium and Chrome use cert9

###
### Requirement: apt install libnss3-tools
###

###
### CA file to install (customize!)
### Retrieve Certname: openssl x509 -noout -subject -in minica.pem
###

certfile="minica.pem"
certname="minica root ca"

###
### For cert8 (legacy - DBM)
###

for certDB in $(find ~/ -name "cert8.db"); do
    certdir=$(dirname ${certDB})
    certutil -A -n "${certname}" -t "TCu,Cu,Tu" -i ${certfile} -d dbm:${certdir}
done

###
### For cert9 (SQL)
###

for certDB in $(find ~/ -name "cert9.db"); do
    certdir=$(dirname ${certDB})
    certutil -A -n "${certname}" -t "TCu,Cu,Tu" -i ${certfile} -d sql:${certdir}
done

Now we need to update our compose file to point to this new Dockerfile and use that instead of the base image. Our docker-compose.yml now looks like this:

services:
  devcontainer:
    build:
      context: .
      dockerfile: Dockerfile
    platform: linux/amd64
    volumes:
      - ..:/workspace:delegated
    ports:
      - "5000:5000"
    environment:
      - POETRY_VIRTUALENVS_IN_PROJECT=true
    command: /bin/sh -c "while sleep 1000; do :; done"
    network_mode: "host"

  azurite:
    image: mcr.microsoft.com/azure-storage/azurite
    ports:
      - "127.0.0.1:10000:10000"
      - "127.0.0.1:10001:10001"
      - "127.0.0.1:10002:10002"
    command: >
      azurite
      --blobHost 0.0.0.0
      --queueHost 0.0.0.0
      --tableHost 0.0.0.0
      --cert /workspace/certs/cert.pem
      --key /workspace/certs/key.pem
      --oauth basic
    volumes:
      - ./azurite-certs/certs:/workspace/certs

Remember to rebuild your devcontainer after making these changes

Configure the local credential for OAuth

You should now be able to run your code and you won’t receive any SSL certificate errors. But the credential provider will most likely complain that it could not find a credential:

[2024-08-30T10:23:59.038Z] DefaultAzureCredential failed to retrieve a token from the included credentials.
[2024-08-30T10:23:59.039Z] Attempted credentials:
[2024-08-30T10:23:59.039Z]      EnvironmentCredential: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
[2024-08-30T10:23:59.039Z] Visit https://aka.ms/azsdk/python/identity/environmentcredential/troubleshoot to troubleshoot this issue.
[2024-08-30T10:23:59.039Z]      ManagedIdentityCredential: ManagedIdentityCredential authentication unavailable. The requested identity has not been assigned to this resource. Error: Unexpected response "{'error': 'invalid_request', 'error_description': 'Identity not found'}"
[2024-08-30T10:23:59.039Z]      SharedTokenCacheCredential: SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache.
[2024-08-30T10:23:59.039Z]      AzureCliCredential: Please run 'az login' to set up an account
[2024-08-30T10:23:59.039Z]      AzurePowerShellCredential: PowerShell is not installed
[2024-08-30T10:23:59.039Z]      AzureDeveloperCliCredential: Azure Developer CLI could not be found. Please visit https://aka.ms/azure-dev for installation instructions and then,once installed, authenticate to your Azure account using 'azd auth login'.
[2024-08-30T10:23:59.039Z] To mitigate this issue, please refer to the troubleshooting guidelines here at https://aka.ms/azsdk/python/identity/defaultazurecredential/troubleshoot.

The solution for this is relatively simple. Azurite only performs basic validation on the presented token – checking for expiry and structure, but does not validate the permissions associated with the token. So we can simply log in with the Azure CLI (az login) to ensure that a principal is available.

Configure Azure Storage Explorer to trust the minica root certificate

Azure Storage Explorer also needs to be configured to trust the new root CA. To do this, click Edit > SSL Certificates > Import Certificates and import the minica.pem file:

Next, reestablish your connection with Azurite and check the Use HTTPS box:

You can access this folder at ~/Library/Application Support/StorageExplorer/certs/ on a Mac. Restart Azurite and you’re good to go!