Bring half of the water up to a boil over high heat. Once it has reached a rolling boil (constant big bubbles), remove the pot from the heat and add in the tea bags and baking soda. Allow tea to steep for 5-6 minutes, but no longer.
After 6 minutes, carefully remove the tea bags from the pot without squeezing the liquid from the bags. Add in the sugar and stir until dissolved.
Add the remaining cold water to the tea mixture and pour it into a pitcher.
Pour tea over ice in a tall glass. Add mint leaves or sliced lemons, as desired.