I love using devcontainers to manage my development environment. They make it super easy to ensure a consistent development stack is in place. Recently i started developing against a MongoDB instance. For node.js, i use mongodb-unit to spin up a standalone server on the local client. But there’s no equivalent package for Python.
Although there are lots of posts on stackoverflow about configuring a single node replicaset using a healthcheck, and there’s even an example given by a MongoDB employee, they didnt work for me. When setting up the server to use authentication, it also needs a keyfile, which has to be generated and secured in a specific way or you get this error:
BadValue: security.keyFile is required when authorization is enabled with replica sets
Without authentication, i was unable to create databases and collections, but the username and password in MONGODB_INITDB_ROOT_USERNAME and MONGODB_INITDB_ROOT_PASSWORD didnt get created automatically in the admin database:
{"t":{"$date":"2023-09-28T20:59:21.342+00:00"},"s":"I", "c":"ACCESS", "id":5286307, "ctx":"conn30","msg":"Failed to authenticate","attr":{"client":"127.0.0.1:33216","isSpeculative":true,"isClusterMember":false,"mechanism":"SCRAM-SHA-256","user":"root","db":"admin","error":"UserNotFound: Could not find user \"root\" for db \"admin\"","result":11,"metrics":{"conversation_duration":{"micros":221,"summary":{"0":{"step":1,"step_total":2,"duration_micros":206}}}},"extraInfo":{}}}
Mongo has a very clear, step by step instructions to set up a replicaset, but it requires a lot of manual steps. So i decided to automate it with a bash script, and then trigger this as a healthcheck. Here are the steps i followed:
My requirements are for an ephemeral database – that means the data is destroyed when the container is removed. To persist the data, you need to map the container folder /data/db to a local folder using a volume in the docker-compose file.
- Planning it out
- Create a script to initialize the replicaset and create the root user
- Create a Dockerfile to generate the replica keyfile and inject my initialisation script
- Create a docker-compose.yaml file to build my app landscape
- Create devcontainer.json to bring it all together
- Accessing from the local machine
Planning it out
Here’s what we’re building. 4 files and at the end, MongoDB will be running and we can connect to it from both our local machine and from inside the devcontainer.
<project workspace>
|
|-- .devcontainer
| |-- devcontainer.json
| |-- docker-compose.yaml
| |-- Dockerfile
| |-- mongodb_healthcheck.sh
|
|-- <other code and folders>
Create a script to initialize the replicaset and create the root user
The script is intended to be idempotent i.e. you can run it several times and it will only return 0 (success) when the replicaset is up and running, and the username/password works:
[Start]
|
[check_replica_set]
|
|--[Yes]--->[check_authentication]--->[Yes]--->[Exit 0]
| |
| |--[No]
| |
| [create_root_user]
| |
| |--[Success]--->[Exit 1]
| |
| |--[Failure]--->[Exit 1]
|
|--[No]--->[initialize_replica_set]
|
|--[Already Initialized or Success]--->[Exit 1]
|
|--[Failure]--->[Exit 1]
Here’s the full script:
#!/bin/bash
# Function to check if MongoDB replica set is ready
check_replica_set() {
echo "init_script: Checking MongoDB replica set..."
local is_master_result=$(mongosh --quiet --eval 'rs.isMaster().ismaster')
echo "Result of rs.isMaster().ismaster: $is_master_result" # Display the result
if echo "$is_master_result" | grep -q true; then
echo "init_script: MongoDB replica set is ready."
return 0
else
echo "init_script: MongoDB replica set NOT ready."
return 1
fi
}
# Function to initialize MongoDB replica set if necessary
initialize_replica_set() {
echo "init_script: Starting to initialise replica set."
local result=$(mongosh --quiet --eval 'rs.initiate()')
echo "Result of rs.initiate(): $result" # Display the result
if [[ "$result" == *"already initialized"* || "$result" == *'ok: 1'* || "$result" == *'"ok" : 1'* ]]; then
echo "init_script: MongoDB replica set is already initialized or initialized successfully."
exit 0
else
echo "init_script: Failed to initialize MongoDB replica set."
exit 1
fi
}
check_authentication() {
local auth_result=$(mongosh -u "$MONGODB_INITDB_ROOT_USERNAME" -p "$MONGODB_INITDB_ROOT_PASSWORD" --quiet --eval "db.runCommand({ ping: 1 })")
echo "Result of authentication: $auth_result" # Display the result
if echo "$auth_result" | grep 'ok' | grep -q '1'; then
echo "init_script: Authentication successful."
exit 0
else
echo "init_script: Authentication failed."
return 1
fi
}
# Function to create MongoDB root user
create_root_user() {
echo "init_script: Creating MongoDB root user..."
output=$(mongosh <<EOF
admin = db.getSiblingDB("admin")
result = admin.createUser(
{
user: "$MONGODB_INITDB_ROOT_USERNAME",
pwd: "$MONGODB_INITDB_ROOT_PASSWORD",
roles: [ { role: "root", db: "admin" } ]
}
)
printjson(result)
EOF
)
echo "Result of createUser: $output" # Display the result
if echo "$output" | grep 'ok' | grep -q '1'; then
echo "init_script: MongoDB root user created successfully."
exit 0
else
echo "init_script: Failed to create admin user."
exit 1
fi
}
# Check if MongoDB replica set is ready and initialize if needed
if check_replica_set; then
if check_authentication; then
exit 0
else
create_root_user
fi
else
initialize_replica_set
fi
Create a Dockerfile to generate the replica keyfile and inject my initialisation script
I wanted to use the existing mongo image for this, and perform the minimum number of changes. So i created a simple Dockerfile which creates a new keyfile, and puts the init script in. I then trigger the init script as a ‘healthcheck’ meaning it’ll get automatically triggered by Docker after 30 seconds, and then at 10 second intervals, for up to 10,000 seconds (!!!):
FROM mongo
# Initiate replica set
RUN openssl rand -base64 756 > "/tmp/replica.key"
RUN chmod 600 /tmp/replica.key
RUN chown 999:999 /tmp/replica.key
# Copy the health check script to the container
COPY mongodb_healthcheck.sh /usr/local/bin/
# Set execute permissions for the script
RUN chmod +x /usr/local/bin/mongodb_healthcheck.sh
# Define the health check command
HEALTHCHECK --interval=10s --timeout=5s --start-period=30s --retries=100 CMD /usr/local/bin/mongodb_healthcheck.sh
CMD ["mongod", "--replSet", "rs0", "--bind_ip_all", "--keyFile", "/tmp/replica.key", "--auth"]
Create a docker-compose.yaml file to build my app landscape
This is pretty simple. I wanted to use the Microsoft Python 3.11 devcontainer, and have it access Mongo in my MongoDB container. I want to set the root username and password here too. I created a network to allow the pods to talk to each other:
version: '3'
services:
app:
image: mcr.microsoft.com/devcontainers/python:3.11
command: ["sleep", "infinity"]
volumes:
- ..:/workspace:cached
ports:
- "5000:5000"
environment:
- "PYTHONBUFFERED=1"
- "PYTHONUNBUFFERED=1"
networks:
- mynetwork
mongodb:
build:
dockerfile: ./Dockerfile
ports:
- "27017:27017"
environment:
- MONGODB_INITDB_ROOT_USERNAME=root
- MONGODB_INITDB_ROOT_PASSWORD=example
hostname: mongodb
networks:
- mynetwork
networks:
mynetwork:
Create devcontainer.json to bring it all together
devcontainer.json is an open standard. The only bit that tripepd me up here was the need to be explicit about which service was exposing which port, which you do by adding the service name (from the docker-compose.yaml file) in front of it:
{
"name": "Python 3.11 + MongoDB",
"dockerComposeFile": "docker-compose.yml",
"workspaceFolder": "/workspace",
"service": "app",
"features": {
"ghcr.io/devcontainers/features/node:1": {
"version": "latest"
},
"ghcr.io/devcontainers-contrib/features/poetry:2": {}
},
"forwardPorts": [
"app:5000",
"mongodb:27017"
],
"customizations": {
"vscode": {
"settings": {
"python.defaultInterpreterPath": "/usr/local/bin/python",
"python.linting.pylintEnabled": false,
"python.linting.flake8Enabled": true,
"python.linting.enabled": true,
"editor.detectIndentation": false,
"editor.tabSize": 4
},
"extensions": [
"ms-python.python",
"ms-python.flake8",
"ms-python.vscode-pylance",
"VisualStudioExptTeam.vscodeintellicode",
"njpwerner.autodocstring",
"GitHub.copilot",
"GitHub.copilot-chat",
"GitHub.copilot-labs"
]
}
}
}
Accessing from the local machine
MongoSH will now connect from the local machine or devcontainer. Without a username/password you get quite limited access, but using mongosh -u root -p example it’ll connect just fine and you can administer the database using the account we created earler. If you just try to connect using MonoDB Compass however you’ll get this error:
getaddrinfo ENOTFOUND mongodb
This can be solved by adding directConnection=true to the connection string e.g.:
mongodb://root:example@127.0.0.1:27017/?directConnection=true&serverSelectionTimeoutMS=2000
