Big news for the PagoPA developer community! 🎉

We're officially moving from Yarn to PNPM for our dependency management. This change is a significant step forward for the PagoPA DX Toolchain, bringing improvements in speed, consistency and security to our development workflow.

Why PNPM?​

You might be asking why we're making this switch. Here are a few key reasons:

• Faster and More Efficient: PNPM's unique approach to dependency management, using a content-addressable store, avoids the need for duplicated packages. This means faster installations and less disk space consumption.

• A Safer Dependency Tree: PNPM creates a strict, non-flat node_modules structure, which helps prevent accidental access to dependencies that are not explicitly declared in your package.json. This makes our dependency trees more predictable and secure. This is particularly beneficial for our monorepo setups, as it will prevent ghost dependencies issues before the deployment.

Getting Started​

We've already updated the PagoPA DX Toolchain to include PNPM support. To make the migration as smooth as possible, we've created a migration script accessible via our DX CLI.

To run the migration script, use the following command in your project's root directory:

npx @pagopa/dx-cli codemod apply use-pnpm

This script will import your existing lockfile and it will automatically replace Yarn and NPM commands in your CI/CD workflows with their PNPM equivalents.

Prerequisites​

• Node.js 20.19.5 or higher with corepack enabled.

  note

  Corepack is included by default with Node.js versions 16.9.0 and above, but you'll need to enable it manually for version 20 and above. You can do this by running:

  corepack enable
  

• Your project should use the PagoPA DX workflows for code review and deployment. The migration script will update only the following workflows:

  • js_code_review.yaml
  • web_app_deploy.yaml
  • function_app_deploy.yaml

After the migration​

After running the migration script, test your project locally to ensure all scripts are working as expected, and all dependencies are correctly resolved.

Quick Guide: Using PNPM​

To help you get started, here's a quick reference guide on how to perform common tasks with PNPM, compared to their Yarn equivalents.

Install dependencies​

  # Install dependencies
  yarn install

  # Don't alter the lockfile
  yarn install --immutable

  # Add a new dependency to root package
  yarn add express

  # Add a development dependency to root package
  yarn add -D typescript

  # Install dependencies
  npm install

  # Don't alter the lockfile
  npm ci

  # Add a new dependency to root package
  npm install express

  # Add a development dependency to root package
  npm install --save-dev typescript

  # Install dependencies
  pnpm install

  # Don't alter the lockfile.
  # In CI is the default behavior, even without this flag.
  pnpm install --frozen-lockfile

  # Add a new dependency to root package
  # By default, it does not allow adding dependencies to root packages
  # unless the -w (workspace) flag is used
  pnpm add -w express

  # Add a development dependency to root package
  pnpm add -w -D typescript

Multi-projects support​

Both Yarn and PNPM support managing multiple projects within a single repository. Yarn requires that packages are defined in the workspaces field of the root package.json, while PNPM uses a dedicated pnpm-workspace.yaml file.

  {
    "name": "my-monorepo",
    "private": true,
    "workspaces": [
      "apps/*"
    ]
  }

  # Add a new dependency to my-app package
  yarn workspace my-app add express

  {
    "name": "my-monorepo",
    "private": true,
    "workspaces": [
      "apps/*"
    ]
  }

  # Add a new dependency to my-app package
  npm -w my-app add express

  packages:
    - 'apps/*'

  # Add a new dependency to my-app package
  pnpm --filter my-app add express

  # Note that filters, supports wildcards and complex expressions
  # Example: Add a new dependency to all packages starting with "my-"
  pnpm --filter "my-*" add express

  # Examples: Install dependencies only in packages that have changed
  # relative to the main branch
  pnpm --filter "...[origin/main]" install

  # The --filter syntax is the same used by Turbo

Run scripts​

Both Yarn and PNPM allow you to run scripts defined in your package.json files.

  # Run the build script in the current package
  yarn build

  # Run the build script in the my-app package
  yarn workspace my-app build

  # Run the build script in all packages
  yarn workspaces foreach --all run build

  # Run the build script in the current package
  npm run build

  # Run the build script in the my-app package
  npm run build -w my-app

  # Run the build script in all packages
  npm run build --workspaces

  # Run the build script in the current package
  pnpm build

  # Run the build script in the my-app package
  pnpm --filter my-app build

  # Run the build script in all packages
  pnpm -r build

Other features​

PNPM offers unique features that help us manage dependencies at scale. Here are some examples.

Catalogs​

Catalogs are a workspaces feature for defining dependency version ranges as reusable constants.

Imagine you have multiple packages that depend on react and react-dom. Instead of specifying the version in each package, you can define a catalog in the root pnpm-workspace.yaml file.

packages:
  - "apps/*"
  - "packages/*"

catalogs:
  react: ^18.0.0
  react-dom: ^18.0.0

Then, in each package's package.json, you can reference the catalog:

{
  "name": "my-app",
  "version": "1.0.0",
  "dependencies": {
    "react": "catalog:",
    "react-dom": "catalog:"
  }
}

{
  "name": "my-library",
  "version": "1.0.0",
  "dependencies": {
    "react": "catalog:",
    "react-dom": "catalog:"
  }
}

When you run pnpm install, PNPM will resolve the catalog: references to the versions defined in the pnpm-workspace.yaml file. This ensures that all packages use the same version of react and react-dom, making it easier to manage and update dependencies across the monorepo.

Security​

PNPM has built-in security features that help protect your projects from vulnerabilities.

• Post Install Scripts Control: PNPM allows you to control the execution of post-install scripts, which can be a vector for supply chain attacks. You can disable these scripts globally or on a per-package basis.
• minimumReleaseAge: This setting allows you to specify a minimum age for package releases. This can help avoid using newly published packages that might not have been thoroughly used yet.

For a complete list of PNPM features and commands, check out the official PNPM documentation.

At PagoPA, we heavily rely on Infrastructure as Code (IaC) practices to deliver our cloud solutions. Terraform has become a fundamental tool in our engineering toolkit, allowing us to manage our infrastructure efficiently and ensure consistency across environments.

As our projects grew in number and complexity, we faced a common challenge: how to effectively share and reuse Terraform code while maintaining quality and consistency. This led us to explore publishing our modules and provider to the Terraform Registry, all while preserving our internal development process based on a monorepo structure.

In this post, we'll share our journey as the Developer Experience team, the challenges we faced, and the technical solutions we implemented to bridge the gap between developing Terraform modules in a single repository and meeting the Terraform Registry requirements.

The new Terraform module Azure GitHub Environment Bootstrap developed by the DevEx team, has finally left the beta status by reaching its first major version release!

This module is useful for anybody that has just created a new repository and wants to focus quickly on their goals rather than spending hours in setting up everything around the new repository. The module focuses on projects which leverage Azure, GitHub, and a single environment (production). After applying this module, the repository will have:

• completed the setup needed to launch GitHub Actions workflows
• a dedicated private GitHub Runner to connect to private Azure resources from GitHub pipelines
• an Azure resource group to deploy resources generally contained in infra/resources
• the required permissions to operate on domain resources
• a secure and smooth configuration

The new Terraform module Azure Service Bus Namespace has been released by the DevEx team to enable customers to easily and straightforwardly set up Service Bus!

This module abstracts inner complexities such as networking, authentication, and scaling. By choosing the l tier (which matches the Premium SKU), the module provides private communication via private endpoints and also includes an autoscaler resource to dynamically change the number of instances depending on CPU and memory metrics.

The new Terraform module Azure Service Bus Alerts has been released to help teams monitor their Azure Service Bus instances with minimal setup and improved observability.

This module allows teams to configure alerts for both active and dead-lettered messages across queues and topics, providing flexibility through customizable thresholds, severities, and metric windows.

The digital services we provide interact through REST APIs that are documented using OpenAPI specifications.

Using API clients requires the code that handles HTTP requests and responses to strictly adhere to the OpenAPI specifications. This is crucial to ensure that the exchanged data is valid and consistent with the defined schemas. Similarly, when providing APIs through NodeJS services, such as Express or similar frameworks, it is important to follow the same standards. This ensures that the APIs are reliable, scalable, and easy to maintain.

Imagine being able to release the first API for a new digital service into production in minutes instead of weeks, having fewer decisions to make, less code to interpret and maintain, onboarding new team members with zero downtime: this is the goal we set for ourselves with the Developer Experience (DevEx) initiative.

At the heart of the Engineering Area, a group of Senior, Cloud, and Staff Engineers has decided to tackle the daily challenges that slow down our work. We're here to break down barriers, simplify processes, and make software development smoother and more rewarding for everyone.