We are proud to announce the launch of Pmbot, a self hosted, extensible, automated dependency update platform.
From idea to proof of concept, design, and a lot of dark magic, our team has dedicated the past 1+ year to making this project come to life. As you are reading these lines, we are officially entering our public beta phase.
How we came to be
Our company was born two years ago. As a small team of two software developers, we manage a lot of Git repositories that we self host on a Gitlab instance. As projects grew, we quickly realized that it would be impossible to keep all of them up to date without an automated solution.
We first started looking at existing technologies but the platforms we found didn't fit the way we envisioned automated dependency updates. There were two main drawbacks to any of them:
- They do not offer (or it was not clear at the time whether they did) a free, self-hosted installation. A platform only available in the cloud is not an option for a company like ours which self-hosts 99% of its infrastructure and cannot rely on third-party applications due to NDAs and privacy issues.
- All of our projects have automated CI workflows that build, test and lint (and more) our code. We never let anything merge to the main branch if CI doesn't pass. They aren't aware of our CI. In most cases, they update the dependencies and then open pull requests. You end up being flooded with PRs and clicking "merge" every time becomes a hassle. If the platform was aware of your CI, it could just merge automatically when the update is successful and open an issue or send you a message when it fails.
- Dependencies are considered separately. This makes sense, but one PR per update is annoying. What we'd rather have is that all successful updates are merged together and the failed ones are left aside. This way, if out of 5 successful only one failed, you still get the 4 successful ones merged.
So, after assessment of the task ahead of us, and ensuring we could afford to live while developing this tool, we jumped in.
Our requirements were the following:
- Core features should be free. We strongly believe in the model that Gitlab adopted: free for all, premium for few. We believe that most tools, whether Open Source or not, should provide a free version that offers sufficient features in their free version so that most people can use it, and offers premium features for those that have specific needs.
- You should be able to self-host it. We are aware that a lot of folks out there prefer cloud solutions, which we do offer, but we believe that it important to allow developers and companies to self-host their tools. It's important to give users a choice, so everyone can be happy. Particularly with recent GDPR and increasing privacy concerns, we wanted to make sure you would have the freedom of choosing what you see fit.
- It should have a minimal footprint in your daily work. No PR flood, only notifications that matter. Plus, this behavior should be fully configurable.
- It should be pluggable. We love software that allows you to extend core functionalities by writing simple plugins. A lot of popular solutions like DroneCI, Ghost, or Wordpress have become popular because they offered a way to extend core feature by plugging your own code. That, to us, mattered too.
Our goal was actually quite simple. We wanted to replicate how we updated manually the dependencies of our Git projects. That is, for a given Git repository:
- clone the Git repository
- create an update branch
- update all dependencies
- check locally that the project compiles and that all tests pass
- if a dependency breaks, try to find why, and fix it
- commit and push
- wait fo CI to pass
- merge into the main branch
Sounds simple, but in practice, let's face the truth, we all know how things go... It's been six months that you haven't touched this project, most (if not all) dependencies are out of date, some have a new major version, your
npm install fails miserably as your terminal displays obscure errors you've never encountered before. You try to update your dependencies and surprisingly, some let you down as APIs have changed, features have been removed, compatibility issues have appeared, semantic versioning has not been enforced and newer minor version contain breaking changes... the list is long, and one thing that is certain, you end up spending up to a few days updating your dependencies and you tell yourself that if you had done this more often, it would have been easier to know which update caused the errors.
Don't worry, been there, done that. That's why we wrote Pmbot, and we hope that it'll change your life as it changed ours.
Of course, a bot can't figure out which dependencies caused an error if it updates all of them at once. The ideal workflow, after cloning and creating the update branch would be to update one dependency at a time and wait for CI to run for that specific update, then move on to the next dependency. A human won't wait for CI to run all night long for each update commit, but a bot will :)
The foundation of Pmbot is actually simple: in 99% of cases, an automated dependency update is possible. Don't bother opening pull requests, Pmbot waits for your CI to pass all checks, so it's safe to merge the updates. And best of all, if one or more updates failed, that's fine, you still get all the successful ones merged in a single commit ! For the 1% left, we'll just send you a message, open an issue on your Gitlab or Github repository, or, even better, we'll let you define what Pmbot should do in that case by using a native plugin or creating a custom one.
Best of all, we have built Pmbot to be compatible with several Git servers, CI platforms and package managers. And, if you can't find your favorite package manager in those we already support, you can write a simple package manager plugin ! Within the next few months, we will be adding support for more, so make sure to stay tuned !
- coming soon: Bitbucket, Gitea...
- Gitlab CI
- Drone CI
coming soon: Github actions, Travis CI...
The start of our (beta) adventure
We have used Pmbot internally for a while, and we think that now is the time to share it with the rest of the world.
We love tools that feel and look good, and we strongly believe that well designed and engineered interfaces make the difference between something you'll like and something you'll love using. So, we've worked hard and collaborated with Pixel Point to create an experience we hope you'll enjoy.
Of course, nothing is perfect, and we know that there is always room for improvement. You may find bugs, you will want more features or to enhance existing ones, and that's great, it's a learning process and the point of a beta phase. We've written Pmbot to fit our needs, but now we'd like to make it fit yours.
Pmbot is currently available in self-hosted or cloud versions. Our community version if free for all, and our cloud version is free for open source projects. You'll find a list of all of our editions with prices and features on our official website.
Whether you are a free user or a paying customer, your opinion matters and we'd love to hear what you think, so we've opened a Github repository dedicated to tracking issues. There, you'll be able to submit feature requests, bug reports and ask questions. Our documentation is also available on Github, so you will be able to improve it as you see fit. Suggestions, improvements and constructive criticism are welcome at all times !
We have never done anything like this before, and we are excited to live this adventure with all of you out there, whether you are a developer, a company, or a group of enthusiasts, passionate about software engineering, automation, and willing to make a difference in the world by providing others with great tools to change their life, for the better.
This is the beginning of a wild adventure that we are excited to share with you. We are a small, self-funded team but we hope to grow as our user base increases over time. If you want to support us, just pick a plan, every subscription helps fund the development of this platform and gives us a chance to grow.
As we go along, we will be adding support for new package managers, CI platforms, and Git providers. We will release new plugins, features, bug fixes, and more. We believe in the Open Source community, and for this reason we promise to contribute when we can. Over the next few weeks, we plan to release a plugin devkit as well as open-sourcing our plugin codebase. We will be improving documentation and will try to simplify installation and project setup.
If you want to stay tuned, follow us on Twitter or subscribe to our blog.
We thank you a lot for reading us and we look forward to having feedback from you !
All the best,
Your beloved Pmbot team.