Not 100 per cent sure I have the right Pea ... Mostly low erect shrub but here a taller one, about 1.5m. Fruits are dark green/purplish before opening to the yellow pea flower. Leaves seem to come in pairs; growing on clay soil.
Can't be sure from these photos. It could also be G. latifolium which gets taller, to 3m in the book, but more usually about 1.5m in my experience. You need a side on photo of an open flower to show whether there is a fringe of short white hairs along the keel (the inside, grey, petal). I think I am getting just a hint of this in your first photo, peeking out between the yellow side petals. If it has an obvious fringe that would make it latifolium. The Flora of NSW says for virgatum, keel sometimes minutely ciliate, so if the white hairs are present at all, they should be very short if it is that species.
I'm afraid it may be too late for these now as I don't see any still in flower ... will look around other areas of the Retreat though, to see if there are any still in flower.
A few flowers remain and I have to say that you have extraordinary eye sight! Yes, a fringe of very short grey hairs along the top of the keel - like 'one day' stubble.
OK, let's say virgatum then. In my photos of latifolium once the flower is open the fringe of white hairs is really obvious. I have one shot of virgatum in which they can be seen but are shorter and much less obvious, which sounds more like your plants.