Yes, you have it. You could also say, "As the day began..." or "When the day began..."
The opposite of "at the end of" is "at the beginning of". To determine the correct phrase, you can consider the context. In this case, since you mention being tired but happy, it suggests that you're referring to a specific point in time, rather than a period of time. Therefore, you would use "at the beginning of" to refer to the starting point of the day.
