As I explained in the last post, I’m trying to cut out a perfect outline around the label in the photographs I’ve taken. My first experiment with GrabCut was underwhelming though. The problem is that the interlocking serrations (of yes! that’s what they are called.*) were being left in.
*No, I didn’t know what they were called either but we had some fun on Twitter coming up with names until the right answer was provided https://twitter.com/BBCRecordsVinyl/status/1615710104719966209
Grab A Bite
As I expected, GrabCut to was not an immediate success. You need a background that has a lot in common and a foreground which is distinct from that. I used this tutorial but hit a snag quite fast. I began by using the bounding box method.
Bound To Work?
The first image I used to test this was the slightly doctored photo with a larger black border added prepared for the unsuccessful circle method. That meant I could reuse the code for finding the circle around the label to define the bounding box (rect) that would tell GrabCut where the foreground image is. There was a problem with this though. Here’s the probable foreground and probable background inside the bounding box.
The problem seems to be that the extra border I added doesn’t match the vinyl around the label. The bottom right corner is closest to being correct but even there the interlocking serrations (love that name!) are still being identified as the foreground image. This is nothing like as good as the Windows Paint 3D result from the same image though. That was a cinch, so what is going wrong?
For the first test, I only passed through one iteration. Multiplying to 15 iterations is an improvement, but the time started to drag out to a minute or more and the results were not nearly close enough to suggest that a few more goes around would be enough.
Masked Hero?
There is another way to use GrabCut. You can pass the mask into the algorithm, and take some of the guesswork out of the shape it’s looking for. I have already found a circle which is very nearly the right size, so maybe if I can use that as the mask GrabCut will fix the rest?
Well, that worked very well. But, that isn’t too surprising. I had removed some of the worst felctiosna from around the label already.
I also had to increase the iterations to get a better result because the text was being mangled and the letter’s bowls filled with black. It settled out out x4 iterations. And I didn’t quite get there. You can see the problems in this Probable Background mask created by GrabCut. The green circle is the Hough circle and inside that the white areas are identified as back ground. It’s good that a lot of the outer reflections are found, but it’s not perfect.
In fact this is no better and actually worse than just using a bitwise mask. Worse because it’s slow and misidentifies the letter shapes as background. What I was hoping for was that the algorithm would detect the real edge of the circular label, but that was a misunderstanding. The images below show the best result I managed so far with an an original, undoctored, image plus border – four iterations of the alogorithm. Note the faint serrations still visible.
To recap, I had to doctor the image a fair bit from the original photo to get a good result. The main issue was finding the circle – impossible without enough border around it. Then I manually cleaned up a line of reflection between the serrations and the label. This was essential to GrabCut working well. Even with the circle mask the extra noise from those reflections are a problem. Those two issues are not solved in my Python script although adding border will be easy. The only good news is that the serrations can be removed as long as the overall reflections are limited
Conclusions
What I was really trying to do failed. GrabCut is good, but the background is not distinct enough for automatic cut of the foreground and even with a mask there are issues. The real problems stem from the photograph, but can’t I magically deal with them somehow?