It's not called the Fortress of Solitude for fun, it's the remains of the life of a Man who has nobody, full of memories and eventually literal pieces of his dead planet. Eventually he finds new people in Bruce, Diana, Dick, Kara, and Lois to let in.