Start development environment

Requirements

  • Node v16.16.0
  • Docker
  • Docker-compose bigger than or equals to v1.26

Getting started

  1. Get the mono repo from GitHub
git clone [email protected]:CrowdDotDev/crowd-postgres.git
  1. Run start script in project directory
bash start.sh

Start script will pull the images, configure the environment and run the docker-compose stack.
App will be available at https://app.localhost

Testing node backend

Running all tests for the backend

npm run test

Running only tests in a folder/file

npm run test -- src/services/__tests__/
npm run test -- src/services/__tests__/activityService.test.ts

Running a single test by description

npm test -- -t 'Should create non existent activity with no parent' src/services/__tests__/activityService.test.ts

Since npm run test command runs jest under the hood, all jest flags/commands can be used.

npm test -- --help

Logging lambda functions

Localstack container prints lambda logs after lambdas finish execution. If you are working on a long-running lambda, to get the lambda logs in realtime, first change the LAMBDA_EXECUTOR environment variable in docker-compose.yaml file to docker and re-run the compose stack

LAMBDA_EXECUTOR=docker (in docker-compose.yaml)
docker-compose down && docker-compose up

Then find spawned lambda container

docker ps
37663766

Localstack uses mlupin/docker-lambda image to spawn new lambda containers. In the photo above, the first container from the list is the spawned integration lambda (e59a23e324bd)

To get the real-time log stream of the lambda

docker logs e59a23e324bd -f

Killing long-running lambda functions

If you want to prematurely kill a lambda, first find the container then kill it by stopping the container.

docker stop e59a23e324bd

Hot reloading and volume binding

By default hot reloading and volume binding will not be set with docker-compose up or bash start.sh commands.

Before setting it up, we need to install packages on the host machine. In /backend directory, run

npm ci

There is an additional docker-compose.dev.yaml that has the overriding configurations for hot reload and volume bindings. Additionally, there's an environment variable that enables the code mounting for lambda functions. First, enable mount coding by setting LOCALSTACK_LAMBDA_MOUNT_CODE = true in the .env file.
Then, starting containers using the command below will enable hot reloading for lambdas and other containers.

docker-compose --env-file ../backend/.env -f docker-compose.yml -f docker-compose.dev.yml up

If some of the volume bindings are not desired (i.e: you want to only develop for few containers but don’t want to bind to others) the volume bindings in docker-compose.dev.yml can be commented out for services that you’re not developing currently.

For example, if you’re developing an API endpoint for the Express app, you can comment out other services to get a speed boost in container start time (volume bindings slow down container start times — big speed differences can occur in macOS docker desktop)

13381338

An example docker-compose.dev.yaml for backend API development. The rest of the containers’ volume bindings are commented out for better performance.

Run bash start.sh dev to start the stack with volume-binding.

Invoking serverless functions

Containers run in their own network and don’t know about the host machine. That is why serverless functions should be invoked from one of the containers, but not from the host. (we can’t use network mode: host because this mode is only supported on linux docker engine)

Since serverless deployer containers and node backend use the same backend container, we can execute an interactive bash in any of them to invoke serverless functions.

First, check the running containers using

docker ps
37663766

Get the id of the main backend container. In this example, it is e4aead827763

Now run an interactive bash on the container with this id

docker exec -it e4aead827763 /bin/bash

Change directory into the desired serverless stack. Let’s say that we want to invoke check_merge coordinator

cd src/serverless/microservices/python/serverless
npx serverless invoke local "--function" "coordinator" "--data" "{\"service\": \"check_merge\"}"

Limitations

Hot reloading for python microservices is currently not working because of serverless-python-requirements plugin. Probably achievable with the hack discussed here

Github webhooks are not implemented locally yet because the endpoint for webhooks are created dynamically with the deployment of serverless API gateway githubWebhook function. Since this dynamic endpoint is only known after the deployment, we’d need to keep the backend waiting for the deployment to be completed. To avoid this wait time, some static Nginx rule to redirect the webhook requests to dynamic endpoints needs to be implemented.


Did this page help you?