Scared that imagining anything but the end counts as imagining the middle

My ultimate end is a happy marriage with my SP. There are, however, things I would like to experience with him NOT with the intent of them being a way to get to the end, simply things I’d like to experience. For example, we are already in a relationship but we currently live in different cities. Due to him being in the military it’s not up to us where he gets transferred to. I’d like to imagine that he gets transferred to my city and we move in together. Would this be considered messing with the middle?