Task 11: Set up a Monorepo

Now that we have made significant progress on both the frontend and backend of our Posts app, it's time to combine them into a monorepo using PNPM workspaces. This approach will enable us to manage both packages within a single repository and leverage benefits such as unified dependencies and simplified cross-package workflows.

What is a monorepo?

A monorepo, short for "monolithic repository", is a software development approach where multiple projects or packages are stored in a single repository. In a monorepo, all the code for different projects or packages is managed together, rather than having separate repositories for each project.

People use monorepos for various reasons:

1. Code Sharing and Reusability: With a monorepo, code can be easily shared and reused across different projects or packages. This promotes code consistency and reduces duplication.
2. Simplified Dependency Management: In a monorepo, all projects or packages share the same set of dependencies. This eliminates version conflicts and makes it easier to manage and update dependencies.
3. Improved Collaboration: Having all the code in one repository fosters better collaboration among developers. It allows for easier code reviews, sharing of knowledge, and encourages cross-team collaboration.
4. Streamlined Continuous Integration/Continuous Delivery (CI/CD): With a monorepo, CI/CD pipelines can be set up to build, test, and deploy all projects or packages together. This ensures that changes in one project do not break the overall system.
5. Simplified Versioning: In a monorepo, versioning can be done at the repository level, making it simpler to track and manage versions across different projects or packages.

Overall, monorepos provide a centralized and efficient approach to managing multiple projects or packages within a single repository. It is particularly common to use a monorepo when different packages or projects are integral parts of the same application. In our case, we are going to place the frontend and backend of the Posts app in a monorepo.

Step 1: Create a Root Folder and Copy the Source Codes

Follow these steps:

• Create a folder called posts-monorepo.
• Inside the posts-monorepo folder, create two subfolders named app and api.
• Copy the React app that we have developed in the posts-app folder to the posts-monorepo/app subfolder. When copying, make sure not to include the node_modules, .git, .github, .vscode, or dist folders.
• Next, copy the Nest.js app that we have developed in the posts-api folder to the posts-monorepo/api subfolder. When copying, make sure not to include the node_modules, .git, .vscode, or dist folders.

Step 2: Add a Unified .gitignore

Notice that there is a .gitignore file in each of the app and api subfolders. Let's delete those files. Instead, we will create a new .gitignore file in the root folder that combines the content of both files:

# General dependencies
**/node_modules/
**/.pnp
**/.pnp.js

# Build outputs
**/out/
**/build/
**/dist/
**/dist-ssr/

# Testing
**/coverage/
**/.nyc_output

# TypeScript
**/*.tsbuildinfo
**/next-env.d.ts

# Environment and config files
*.pem
**/.env*.local
.env
.env.dev
.env.prod
.env.test

# Logs
npm-debug.log*
pnpm-debug.log*
yarn-debug.log*
yarn-error.log*
lerna-debug.log*
**/*.log
**/logs

# OS files
.DS_Store

# Editor configurations and project files
**/.idea
**/.project
**/.classpath
**/.c9/
**/*.launch
**/.settings/
**/*.sublime-workspace
.vscode/*
!.vscode/settings.json
!.vscode/tasks.json
!.vscode/launch.json
!.vscode/extensions.json

Note: The **/ notation is a glob pattern used in the .gitignore file. It represents a recursive wildcard that matches any directory.

For example, the pattern **/node_modules matches the node_modules directory and all its instances in the subdirectories of the repository. In this case, it would match the node_modules directory in both the app and api subfolders of the posts-monorepo root folder.

Step 3: Vetting the .env Files

We do not use a unified .env file at the root level, although you may see this in some monorepos. Instead, each application has its own .env file placed in its respective subfolder. It's worth mentioning that we haven't used .env for the frontend React app yet, but we have used it for the Nest.js app.

First, make sure you have copied your .env file for the Nest.js app to the api folder. For your reference, these variables must be defined in that .env file:

NODE_ENV=
DB_HOST=
DB_PORT=
DB_USER=
DB_PASSWORD=
DB_NAME=
JWT_SECRET=
JWT_EXPIRATION=

To specify the URL of the frontend React application, we will add another variable called CLIENT_URL to the .env file. As a reminder, our React app is bootstrapped with Vite and it runs on http://localhost:5173. Therefore, you can add this URL as the value for CLIENT_URL in your .env file:

CLIENT_URL=http://localhost:5173

Next, create a .env file inside the app folder, and include the following content:

NODE_ENV=development
VITE_API_URL=http://localhost:3000

In the .env file of the React app (app folder), the VITE_ prefix is added to VITE_API_URL. This is because Vite, the build tool used for the React app, requires variables to be prefixed with VITE_ in order to be accessible during the build process. This convention is specific to Vite and helps distinguish between environment variables used by Vite and other variables used within the React app.

Notice that in the .gitignore file, we exclude the .env file (and its variations) where we store our environment variables. While we do not want to commit the .env file into our version control system, it is a good practice to provide an example of it for other developers to set it up accordingly. This example can be provided in the software documentation, in the repository README, or more commonly, in a .env.example file.

Let's add a .env.example file to the app folder:

NODE_ENV=
VITE_API_URL=

Also, add a .env.example file to the api folder:

NODE_ENV=
DB_HOST=
DB_PORT=
DB_USER=
DB_PASSWORD=
DB_NAME=
JWT_SECRET=
JWT_EXPIRATION=
CLIENT_URL=

With this in place, we will next update our README files to draw developers' attention to these .env.example files.

Step 4: Add a Unified README.md

Delete the README.md files in the app and api folders. Instead, add the following README to the root folder:

# Posts Monorepo

This is a monorepo that contains the client and server applications for the Posts project.

- The client is a React app bootstrapped with Vite.
- The server is a Nest.js application.

## Posts - A Simplified Social Media Platform

Posts is a minimalistic clone of <https://posts.cv/>, which is itself a simplified and scaled-down alternative to X (formerly known as Twitter). We built this clone solely for educational purposes. The following features are planned (and some are already implemented):

- User authentication
- Ability to create text, image, or combined posts
- Ability to follow other users
- Ability to like and comment on posts
- "Highlights" tab: posts from those you follow
- "Everyone" tab: all posts in reverse chronological order

## Getting Started

To get started with this project, follow these steps:

1. **Prerequisites**: Make sure you have Git, Node, and PNPM (the new package manager for Node) installed. If you don't have PNPM, you can install it globally with `npm install -g pnpm`. Additionally, you need to have Docker set up and running to spin up a local Postgres server. If you're new to Docker, you can refer to [this helpful guide](https://docs.docker.com/get-started/).
2. **Repository Setup**: Clone the repository and navigate to the root folder in the terminal.
3. **Dependencies**: Run `pnpm install` to install the dependencies for both the client and server.
4. **Environment Configuration**: Add a `.env` file in each the `app` and `api` sub-folders, similar to their respective `.env.example` files, and fill in the required environment variables.
5. **Database Setup**: Run `pnpm docker:up` to initialize the Postgres server.
6. **Run Locally**: To start the server, run `pnpm start:api`. To start the client, run `pnpm start:app`. Alternatively, you can run `pnpm start:all` to start both the client and server applications.

Notice that the instructions refer to running pnpm install in the root folder to install the dependencies. This brings us to the next step, which is about PNPM workspaces!

Step 5: Create a PNPM Workspace

In this step, we will set up a PNPM workspace, which is a feature built into PNPM to support monorepos. With a workspace, you can unite multiple projects inside a single repository.

PNPM Workspace

PNPM workspace allows you to manage a monorepo, where each project is considered a "package" within a single repository. This approach offers several advantages, including:

1. Unified Dependencies: PNPM workspace allows all packages within the monorepo to share the same set of dependencies. This eliminates duplication of dependencies and reduces the overall size of the project.
2. Simplified Development: PNPM workspace provides a unified command-line interface to manage all packages. You can run scripts, install dependencies, and perform other tasks across multiple packages with a single command.
3. Efficient Development Workflow: PNPM workspace enables you to easily make changes to multiple packages and test them together. This streamlines the development process and improves productivity.

Set up PNPM Workspace

To set up PNPM workspace in our monorepo, follow these steps:

• Delete the pnpm-lock.yaml files in the app and api folders.

• Add a pnpm-workspace.yaml file to the root folder with the following content:

  packages:
    - 'api/**'
    - 'app/**'
  

  The pnpm-workspace.yaml file is a key component to configure PNPM workspace in a monorepo. In this specific case, the file is defining the workspace packages by specifying the directories where the packages are located.

• Let's clean up the package.json file in the api folder:

  {
    "name": "posts-api",
    "version": "0.0.1",
    "private": true,
    "license": "UNLICENSED",
    "scripts": { ... },
    "dependencies": { ... },
    "devDependencies": { ... },
    "jest": { ... }
  }
  

  Note that we are not making any changes to the scripts, dependencies, devDependencies, and jest configurations. Simply make sure the name, version, private and license attributes are correct, and no other attributes are defined.

• Let's clean up the package.json file in the app folder:

  {
    "name": "posts-app",
    "version": "0.0.1",
    "private": true,
    "license": "UNLICENSED",
    "type": "module",
    "scripts": { ... },
    "dependencies": { ... },
    "devDependencies": { ... }
  }
  

  Note that we are not making any changes to the scripts, dependencies, and devDependencies configurations. Simply make sure the name, version, private, license, and type attributes are correct, and no other attributes are defined.

• First, add the following package.json to the root folder:

  {
    "name": "posts",
    "scripts": {
      "docker:up": "pnpm --filter posts-api run docker:up",
      "start:api": "pnpm --filter posts-api run start:dev",
      "start:app": "pnpm --filter posts-app run dev",
      "start:all": "concurrently \"pnpm run start:app\" \"pnpm run start:api\""
    },
    "devDependencies": {
      "concurrently": "^8.2.2"
    }
  }
  

  This package.json file in the root folder of the monorepo serves as the main configuration file for managing the entire project.

  The --filter flag is used with the pnpm command to specify the package or workspace to target. In this case, it is used to run specific scripts in either the posts-app (React app) or the posts-api (Nest.js app) package. These names correspond to the names declared in the package.json file of each package (app and api).

  For example, the start:app script is defined as pnpm --filter posts-app run dev. This means that when you run pnpm run start:app, it will execute the dev script in the posts-app package as defined in app/package.json.

  Similarly, the start:api script is defined as pnpm --filter posts-api run start:dev. This means that when you run pnpm run start:api, it will execute the start:dev script in the posts-api package as defined in api/package.json.

  The start:all script uses the concurrently package, which allows you to run multiple commands concurrently. In this case, it is used to run both the React app and the Nest.js app simultaneously. When you run pnpm run start:all, it will execute both the start:app and start:api scripts concurrently.

  Please note that the provided package.json assumes the presence of the concurrently package as a devDependency.

• Next, open the terminal and ensure that your working directory is the root of the project, posts-monorepo. Then, run pnpm install to install the dependencies for both the frontend (app) and backend (api) servers.

• Once the installation is complete, you will find a pnpm-lock.yaml file in the root folder (but not in the app or api subdirectories). Additionally, there will be a node_modules folder in the root directory, as well as app and api subfolders. The one in the root folder contains the shared dependencies, while the ones inside the subfolders contain the dependencies specific to each package (app and api).

Step 6: Add a Unified Prettier

Currently, the package.json file at the root folder provides a minimal configuration for managing the entire project. Depending on the nature of the project, you may choose to keep this configuration minimal or expand upon it.

On one hand, having a unified configuration promotes consistency and simplifies the management of the entire workspace. On the other hand, keeping the configurations separate allows for modularity in the codebase, making it easier to maintain and update each of the app and api packages in the future.

For example, both the app and api projects utilize Typescript and Eslint. Each project has the necessary packages installed as dependencies and provides separate configurations. While it may seem convenient to unify these configurations in the root folder, it is not advisable. The frontend (app) and backend (api) have different linting rules and TypeScript compiler options. Combining these configurations at the root level can result in conflicts and inconsistencies between the two projects.

💡 It is generally recommended to keep ESLint and TypeScript configurations separate for each project within a monorepo. This approach allows for better flexibility, modularity, and maintainability of the codebase.

Similarly, both the app and api projects use Prettier for code formatting. Each project has Prettier as a dependency and defines its own configuration. To maintain a consistent code style across all packages, it is more sensible to establish a unified Prettier configuration. By having a single Prettier configuration at the root level, we ensure that both the frontend and backend code adhere to the same formatting standards.

Let's set up a unified Prettier configuration:

• Open the terminal and navigate to the api folder. Then, execute the following commands to remove Prettier as a dependency and its configuration file(s):

  rm .prettierrc
  pnpm remove prettier
  

• Next, navigate to the app folder in the terminal and run these commands to remove Prettier as a dependency and its configuration file(s):

  rm .prettierignore .prettierrc.json
  pnpm remove prettier
  

• Open the api/package.json file and remove the following line from it:

  "format": "prettier --write \"src/**/*.ts\" \"test/**/*.ts\"",
  

• Open the app/package.json file and remove the following line from it:

  "prettier": "prettier --write \"src/**/*.{ts,tsx}\" --config \".prettierrc.json\"",
  

• In the root of your monorepo (posts-monorepo), create a .prettierrc file with the following content:

  {
    "semi": true,
    "trailingComma": "all",
    "singleQuote": false,
    "printWidth": 80,
    "tabWidth": 2,
    "endOfLine": "auto"
  }
  

  This configuration sets the rules for code formatting, such as the usage of semicolons, trailing commas, quotes, etc.

• Modify the package.json file in the root folder to include commands for formatting:

  "scripts": {
    "format:write": "prettier --write \"**/{src,test}/**/*.{ts,tsx}\" --config \".prettierrc\" --cache",
    "format:check": "prettier --check \"**/{src,test}/**/*.{ts,tsx}\" --config \".prettierrc\" --cache",
    // other scripts...
  }
  

  These commands will format and check all TypeScript files in the src and test directories of both the app and api packages.

• Then, install Prettier at the root level of your monorepo. Open the terminal at the root folder and execute the following command:

  pnpm add prettier --save-dev --workspace-root
  

• To format your code, run pnpm run format:write from the root directory.

• To check if the files are formatted correctly, run pnpm run format:check.

By following these steps, you'll have a unified code formatting setup across your entire monorepo, ensuring that all code changes, whether in the frontend or backend, adhere to a consistent style. This setup enhances code readability and quality, making it easier for the team to maintain and contribute to the project.

Step 7: Running the React App and the API Server

To ensure the React and Nest.js apps are functioning correctly, follow these steps to run them. Since we have not yet integrated these applications, we will run them separately and verify their proper functionality. This step is crucial because changes have been made to the project structure of each app, and certain settings or configurations may need to be updated.

1. First, run the docker container of the Postgres database (if it is not already running) by executing the pnpm docker:up command.

2. Then, run the backend by executing the pnpm start:api command. Open Postman and test all the endpoints and available operations, ensuring that the protected routes are secured.

3. Next, run the frontend by executing the pnpm start:app command. Open the browser and visit http://localhost:5173/ to verify that the app works as expected. If a specific base attribute has been added in the vite.config.ts file to deploy the app on GitHub, make sure to change it back to the default configuration provided below:

   import path from "path";
   import react from "@vitejs/plugin-react";
   import { defineConfig } from "vite";
   
   export default defineConfig({
     base: "/",
     plugins: [react()],
     resolve: {
       alias: {
         "@": path.resolve(__dirname, "./src"),
       },
     },
   });
   
   

   Once the app is running, you should be able to add and delete posts. Open the browser console and ensure that no errors or debugging messages are printed. Also, keep an eye on the VSCode terminal to ensure there are no errors or issues while working with the app.

4. Finally, stop both applications and run them concurrently one last time using the pnpm start:all command.

If all goes well, you are ready for the next step!

Step 8: Add .vscode Folder

Let's configure the debugging for both the frontend and backend applications. Start by adding a .vscode folder to the root folder (posts-monorepo) of your monorepo. Then, create a launch.json file inside the .vscode folder with the following content:

{
  "version": "0.2.0",
  "configurations": [
    {
      "type": "chrome",
      "request": "launch",
      "name": "Launch Chrome against localhost",
      "url": "http://localhost:5173/",
      "webRoot": "${workspaceFolder}/app/src",
      "sourceMapPathOverrides": {
        "webpack:///src/*": "${webRoot}/*"
      }
    },
    {
      "type": "node",
      "request": "launch",
      "name": "NestJS Debug",
      "skipFiles": ["<node_internals>/**"],
      "program": "${workspaceFolder}/api/src/main.ts",
      "runtimeExecutable": "pnpm",
      "runtimeArgs": ["--filter", "posts-api", "run", "start:debug"],
      "outFiles": ["${workspaceFolder}/api/dist/**/*.js"]
    }
  ]
}

These configurations allow you to debug the frontend (app) and backend (api) packages separately.

To debug the frontend, first run the application using either pnpm start:all or pnpm start:app if the backend is not required. Then, in VSCode, go to the Debug view by clicking on the Debug icon in the Activity Bar on the side of the window. Select Launch Chrome against localhost from the dropdown at the top of the Debug view. Click on the green play (▶️) button to start debugging. A new Chrome instance with debugging enabled should open. You can set breakpoints in your React code (app folder) within VSCode. When the code execution reaches those points, it will pause, allowing you to inspect variables, the call stack, and more.

To debug the backend, stop the application and let the debugger run it. In VSCode, go to the Debug view by clicking on the Debug icon in the Activity Bar. Select NestJS Debug from the dropdown at the top of the Debug view. Click on the green play (▶️) button to start debugging. VSCode will run the backend in debug mode by executing pnpm --filter posts-api run start:debug. You can open up Postman and make a request. You can set breakpoints in your Nest.js code (api folder) within VSCode. When the code execution reaches those points, it will pause, allowing you to inspect variables, the call stack, and more. If you want to work with the frontend while debugging the backend, open a new terminal and run pnpm start:app to run the frontend while the backend is running in debug mode.

The .vscode folder contains configuration files for Visual Studio Code. More files can be added to ensure a standardized development experience across a team of developers.

For instance, you can add an extensions.json file with the following content:

{
  "recommendations": [
    "ms-vscode.js-debug-nightly",
    "dbaeumer.vscode-eslint",
    "mhutchie.git-graph",
    "heybourn.headwind",
    "esbenp.prettier-vscode",
    "kalimahapps.tailwind-config-viewer",
    "bradlc.vscode-tailwindcss",
    "austenc.tailwind-docs"
  ]
}

This file lists recommended extensions for VS Code that can enhance the development experience. It includes extensions for debugging, ESLint integration, Git visualization, Tailwind CSS support, and more. Including this file ensures that all team members have access to helpful tools and maintain a consistent coding environment.

Another example is the settings.json file, which can be added with the following content:

{
  "window.autoDetectColorScheme": true,
  "workbench.preferredLightColorTheme": "Default Light+",
  "workbench.preferredDarkColorTheme": "Default Dark+",
  "workbench.colorTheme": "Default Light+"
}

This file contains editor settings such as color schemes and other preferences.

While it is beneficial to standardize certain settings, individual developers may have different preferences for their local development environment. To avoid conflicts and allow personal customization, you may consider excluding the settings.json file from version control. This can be achieved by removing the ! from !.vscode/settings.json in the .gitignore file.

Step 9: Initialize a Git Repository

As a final step, you need to initialize a Git repository in the root folder, posts-monorepo:

git init -b main
git add .
git commit -m "Initial Commit"

Please note that we haven't set up any deployment strategies for the frontend or backend applications yet. We will address that in the future. For now, our focus will be on integrating the backend and frontend.