Python Workspaces (Monorepos)

Managing Multiple Projects Efficiently

I have already written about monorepos in Flutter, but finally I have found a good solution for Python as well. In this article, I will share how to set up and manage Python workspaces (monorepo) effectively with UV.

Why To Use Workspaces?

When working on multiple related Python projects, managing them in a single repository can offer several advantages:

1. Simplified Dependency Management: Workspaces allow you to manage dependencies for multiple projects in a centralized manner, reducing the risk of version conflicts and simplifying updates.

2. Improved Collaboration: Workspaces facilitate collaboration among team members by providing a unified structure for related projects, making it easier to share code and resources.

3. Streamlined CI/CD Pipelines: With a monorepo setup, you can create more efficient CI/CD pipelines that can test and deploy multiple projects simultaneously.

4. Easier Refactoring: When projects are organized within a single workspace, refactoring code that spans multiple projects becomes more manageable.

However, managing multiple projects in a single repository can also introduce challenges, such as increased complexity in version control and potential performance issues with large repositories. It’s essential to weigh the benefits against the drawbacks based on your specific use case.

The main downside is, if you are under time pressure and need to save one project quickly, one simple change in the repo can result in a whack-a-mole of issues in other projects.

Personally, I usually develop similar kinds of projects, so having them in one repo makes sense for me. I can reuse code, share configurations, and manage dependencies more efficiently. If I make one change, I can test it across all projects easily.

Setting Up Folder Structure

The article will be describing project structure as below:

python-workspaces/
├── pyproject.toml                    # Root workspace configuration
├── packages/
│   ├── package-1/
│   │   ├── pyproject.toml
│   │   └── src/
│   │       ├── __init__.py
│   │       └── main.py
│   ├── package-2/
│   │   ├── pyproject.toml
│   │   └── src/
│   │       ├── __init__.py
│   │       └── main.py
│   └── package-3/                   # (excluded from workspace)
│       ├── pyproject.toml
│       └── src/
│           ├── __init__.py
│           └── main.py
├── plugins/
│   └── plugin-1/
│       ├── pyproject.toml
│       └── src/
│           ├── __init__.py
│           └── main.py
└── projects/
    ├── project-1/
    │   ├── pyproject.toml
    │   └── src/
    │       ├── __init__.py
    │       └── main.py
    └── project-2/
        ├── pyproject.toml
        └── src/
            ├── __init__.py
            └── main.py

We have 2 projects, 2 packages, and 1 plugin in this workspace structure. One package is excluded from the workspace for the sake of demonstration.

If you do not have UV installed, you can find instructions here. Highly recommend it for managing dependencies and virtual environments too!

pyproject.toml Configuration

Root pyproject.toml

Firstly, at the root of the workspace, create a pyproject.toml file to define the workspace and include the packages and projects you want to manage:

[build-system]
requires = ["uv_build>=0.9.5,<0.10.0"]
build-backend = "uv_build"

[tool.uv.workspace]
members = ["packages/*", "plugins/*", "projects/*"]
exclude = ["packages/package-3"]

It is important to use build-system as uv_build to leverage UV’s build capabilities. Otherwise, UV and your IDE will not recognize the workspace structure.

[tool.uv.workspace] section defines the workspace members and excludes package-3 from being part of the workspace.

Folders packages, plugins, and projects contain individual Python projects, each with its own pyproject.toml file.

Package/Project pyproject.toml

Each package, plugin, or project within the workspace should have its own pyproject.toml file to define its specific dependencies and configurations. Here’s an example for package-1:

[project]
name = "package-1"
version = "0.1.0"
description = "A sample package in the workspace."
requires-python = ">=3.12"
dependencies = ["pydantic>=2.0.0"]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

This config defines the individual package’s metadata and dependencies.

Package/Project do not need to use uv_build as build backend, only the root workspace pyproject.toml file.

If we want to make package-2 depend on package-1, we can specify it in the dependencies section like this:

[project]
name = "package-2"
version = "0.1.0"
description = "Another sample package depending on package-1."
requires-python = ">=3.12"
dependencies = ["package-1", "numpy>=1.24.0"]

[tool.uv.sources]
package-1 = { workspace = true }

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

Here, we added package-1 to the dependencies and specified in [tool.uv.sources] that package-1 is part of the workspace.

Similarly, you can set up pyproject.toml files for other packages, plugins, and projects in the workspace.

For example, in project-1’s pyproject.toml, you might have:

[project]
name = "project-1"
version = "0.1.0"
description = "A sample project in the workspace."
requires-python = ">=3.12"
dependencies = [
    "fastapi>=0.100.0",
    "uvicorn>=0.38.0",
    "package-1",
    "package-2"
]

[tool.uv.sources]
package-1 = { workspace = true }
package-2 = { workspace = true }

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

After this setup , you can navigate to the root of the workspace and run:

uv sync

This command will synchronize the dependencies across all projects in the workspace and create virtual environments as needed.

When the synchronization is complete, you can run:

uv run projects/project-1/src/main.py

And you should see the output from project-1.

If you use VSCode with Pylance, it can lint errors on the imports, because it does not understand the workspace structure defined by UV. You can read more about it here.

Adding new Packages/Projects

You can easily add a new package/project just by using the following command:

uv init --package packages/package-4

Conclusion

More and more projects are using monorepos/workspaces to manage multiple related projects in a single repository.

The main benefit is that you are forced to maintain compatibility across projects, which can lead to better code quality and easier maintenance.

Mainly, I finally do not have to deal with multiple repositories at github because I can manage all my related projects in one place.

Big thanks go to creators of UV, and it can be found here.

Socials

Thanks for reading this article!

For more content like this, follow me here or on X or LinkedIn.

Subscribe for more