"Potential for racial offence is not always an easy thing to gauge," said the BBC in a statement.
"In this case, there was nothing derogatory of black people in the language used."
"Nevertheless, it is clear that some listeners did infer a derogatory intention," it added.
"We regret this. With hindsight, we believe it would have been preferable to omit the item from the programme because of the risk that it could be misconstrued."
'Shrivel up'
During the discussion, which was recorded at the Chilcompton Gardening Club in Somerset last October, the panellists giggled as they discussed the plant.
Bob Flowerdew admitted he had "only ever seen one close up - and not that colour".
Anne Swithinbank claimed: "I've never seen one in my life...They don't really like the cold, as you can imagine.
"They shrivel up and look very unhappy."
At the time, producer Trevor Taylor went on the station's Feedback programme to defend the segment, saying innuendo had "been a part of Radio 4 for decades".
The editorial complaints unit did not agree with listeners who said it was inappropriate to air the segment at a time when large numbers of children might be listening, as youngsters only formed a small proportion of the audience.