A lot of free software projects do release candidates, so you’ll see 1.9-rc1, 1.9-rc2, 1.9-rc3, …
I have never thought these were a good idea.
You are relying on getting testers, but getting people to test things is difficult (because it’s boring and hard). When you finally release 2.0
, it’s like you’re declaring there are no bugs. Given that (a) software is intractably hard to test and (b) it’s hard to get testers anyway, it’s unlikely that 2.0
will be bug free.
libguestfs takes what I think is a better approach. In 5 days time I will fork whatever we have in development and I will declare it stable branch, release 1.12.0
. That’s not going to be a stable version, and we don’t pretend it is.
Over the days, weeks and months that follow, we backport patches to the 1.12 branch, following rigorous rules. This makes the stable branch more stable over time. It converges on greater and greater stability as we fix more bugs, more conservatively. In libguestfs, some branches get fixes for 7 months and counting (git shortlog).
The elements that I think are important:
- Don’t do release candidates
- Don’t rely on getting testers
- Do use stable branches
- Don’t develop on the stable branch (develop on the head and backport)
- Do have strong rules about what you backport
- Do backport conservatively, and be more conservative over time
- Do use git cherry-pick!
Richard WM Jones 
1. Don’t do release candidates
2. Don’t rely on getting testers
3. Do use stable branches
4. Don’t develop on the stable branch (develop on the head and backport)
5. Do have strong rules about what you backport
6. Do backport conservatively, and be more conservative over time
7. Do use git cherry-pick!
====================
In the last 20 years, I have seen the above mantra many many times, but in asking what it means.. I have never gotten either a consistent answer and even have had conflicting answers. So how does one do the above? [And will any one agree with it :)?]
We branch, and backport stable fixes into the branch over time using git cherry-pick. I’m not sure I can explain it more than that unless you’ve got specific questions.
We do release candidates for software that I work on and we get lots of testers using it. We even get people testing our betas. On another product at our same company, no one is testing the betas. So we stopped releasing them for that product. So I think it just depends on your product and your user base whether it’s useful for you.
How do you quantify what your testers are doing? eg. Do you measure the numbers of downloads during the testing period vs normally (and does it increase)? Do you include debugging or coverage code in your RCs that sends feedback to you, so you can tell how much coverage your testing has had?
But then… why bother using the releases? That code lags behind development in features and if you’re not going to stabilize the code, there’s no difference between using release and dev code.
I’m not sure if I would be happy using that project 🙂
Stable branches give people safe bug fixes on an otherwise frozen featureset.
That is useful because you don’t want to suddenly upgrade and find that a whole bunch of things you rely on have stopped working, or that some feature you need has been reimplemented and is more buggy than the old implementation. It’s the classic use case, for the same people who buy RHEL. It’s also useful for anyone developing software on top.
We are going to stabilize the code, over time.
We develop SSSD this way. Works quite nicely.
The typical user (a RHEL/CentOS user) would go with the stable branch and fall back (fall forward?) to the latest release in order to get a feature/bugfix we decided not to backport, usually.
So you’re really just dealing with terminology… “1.12.0” just translates to “1.12.rc1”?
No. Unless you know of a project that has release candidates rc1, rc2, …, rc14, … going on for 8 months, and continues development at the same time.
Stable branches are completely different things.
Pingback: What’s the scam here? | Richard WM Jones