They currently sit in third place in the table and Holley knows they must improve in their final three games of the season.
"We haven't performed in these last two weeks as we can," he said.
"Tonight was always going be to tight.
"But I just felt that we could have done enough but we didn't take our chances when we had the field position.
"It is two points dropped for us. I'm disappointed.
"We never felt as though we could shake them off, but, fair play to them, they tested us.
"Both sets of players knew how important a win would be and I think we've all come off a bit disappointed with two points each."
With three games remaining the Ospreys are currently third in the table and face a trip to Newport to face the Dragons in their next Magners League match.