Just up from the Brown Bay Trail head theres a small trail up the hill on your right that leads to the ruins. Four different sections with a lookout up top and more overgrown ruins surrounding.
Exact pin, directions, and updates live in the CheeHuu app.